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I. INTRODUCTION 



In order to insure effective manning of newly created jobs and tasks, 
estimates are needed of the capabilities and skills of operator personnel 
together with prescriptions for the manner in which they should be trained 
Similarly, as existing equipments and procedures are modified, estimates 
are required of the impact of such modifications on operator performance 
in order to update selection and training decisions. In both of these 
cases, accurate estimates depend upon a method for systematically trans- 
lating information about the job or task to be perfortried into quantitative 
information on selection and training needs. 

Traditionally, the burden of such translation has fallen upon task 
analysis, a general procedure typically consisting of two distinct aspects 
a description of the salient characteristics of tasks and the translation 
of these characteristics into hypotheses stating the operator capabilities 
necessary for successful performance. While a variety of task-descriptive 
and task-analytic methods have been developed to aid in this translation, 
their effectiveness has been limited by the lack of task-descriptive 
languages which would facilitate inferences about desirable operator 
capabilities given information about salient features of the task. 

In response to this and a series of allied problems, Fleishman and 
his associates undertook the development of several different but inter- 
active taxonomies for the description and classification of tasks (e.g., 
Fleishman, Kinkade, & Chambers, 1968; Fleishman, & Stephenson, 1970; 
Fleishman, Teichner, & Stephenson, 1970; Theologus, Romashko, & Fleishman, 
1970; Wheaton, Mirabel la, & Farina, 1971; Levine, Romashko, & Fleishman, 
1971; Farina, & Wheaton, 1971; and Wheaton, & Mirabel la, 1972). While 
several taxonomies were investigated, the two on which most extensive 
research was conducted provided for detailed descriptions of tasks in 
terms of: human abilities hypothesized as essential to effective task 
performance; and salient or critical goal-, Stimulus-, procedural-, and 
response-dimensions of tasks. 
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The first language uses empirically defined human abilities as its 
vocabulary. In this context, an ability refers to a relatively enduring 
characteristic of an individual which has been inferred from behavioral 
consistencies (e.g., correlations) on several kinds of tasks. Since the 
abilities are empirically defined (through factor-analytic procedures) 
by response consistencies on tests of known reliability, operational 
definitions of abilities are available. Application of these definitions 
in the description and classification of diverse tasks has been reasonably 
successful. Given verbal defiriitions of abilities derived from the 
empirical findings, Theologus, Romas hko, & Fleishman (1970), Theologus, & 
Fleishman (1971), and Levine, Greenbaum (Kramer), & Notkin (1973) have 
demonstrated that experienced analysts can agree on the specification of 
sets of abilif'es presumably related to individual differences in performance 
on a broad cross section of tasks. 

As the complement to description of relevant operator abilities, the 
second language— task characteristics—was designed to provide for the 
description of tasks in terms of intrinsic properties common to a wide 
variety of tasks including goals, procedures, and stimulus-response modes. 
The decision to attempt description in these rather morphological terms, 
instead of using more behavioral-, process-, or ability-oriented descriptors, 
stemmed from a conviction that tasks, in their own right, represented a 
potent class of independent variables contributing to variance in performance. 
This position was presented in terms of an heuristic model of performance 
known as POET (Farina, & Wheaton, 1971). The model simply stated that any 
obtained measure of performance (P) was necessarily a function of at least 
three major classes of variables. These included the specific operator (0) 
whose performance was monitored, the environmental conditions (E) under 
which performance occurred, and the particular task (T) on which performance 
was measured. The implication of the model was that if the variables com- 
prising a task were manipulated singly or in combination (e.g., creating 
a number of similar but morphologically different tasks), the resultant 
effects on performance and on the abilities related to individual differ- 
ences in performance might then be mapped systematically. 



Considered jointly: cherefore, the task-characteristics and abilities 
languages furnished a conceptual basis for translating information about 
the task to be performed into information about the kinds of operator 
capabilities and skills required for successful performance. Hypothetically. 
task analysts would not only be able to describe the features of a given 
task and to infer the kinds of abilities required, but they would also 
be able to specify changes in ability requirements as selected features of 
the task were manipulated. These kinds of specifications were viewed as 
central to improved selection and training decisions. 

The program of research summarized in the present report was designed 
to investigate the task-analytic issues raised above at a more fundamental 
level than had been undertaken heretofore. Its goal was essentially three- 
fold: 1) to determine whether presumably small but systematic variations 
in tasks were reflected in changes in ability requirements; 2) to establish 
whether such changes were orderly, permiisting the development of principles 
relating task variations to ability requirements; and 3) to extend the 
study of basic human abilities to more cognitively-oriented tasks. In 
essence, therefore, the research program has been concerned with an 
empirical elaboration of the interplay between task demands and ability 
requirements conceptualized in earlier efforts. 

The approach adopted to address these issues has entailed the investi- 
gation of classes of tasks both representative of those prevalent in the 
modern Navy and of theoretical interest. In each instance performance 
data are obtained for large samples of subjects who perform on systematically- 
induced variations of the task of interest. Empirically established ability 
profiles for these same subjects are then related to the performance data 
in a variety of ways to ascertain changes in ability requirements. The 
resultant patterns of changes are then interpreted for meaning. 

The specific results and more general findings stemming from appli- 
cation of this approach to three different classes of tasks (e.g., auditory 
signal identification, electronic troubleshooting, and problem solving) 
are presented and summari2:ed in the following sections of this report. 
In Section II, the basic method is elaborated and results are presented 

3 



ERLC 



13 



concerning: 1) the effect of task manipulations on performance; 2) the 
identification of operator abilities; and 3) the investigation of changing 
ability requirements in response to task manipulations. In Section III, 
intervening variables descriptive of the subjects are explored as mechanisms 
for consolidating and systematizing some of the complex changes in ability 
requirements which were demonstrated. Relevant issues include: 1) the 
kind of problem-solving strategies used by different subjects; 2) the 
relation between the kind of strategy employed and the level of perfor- 
mance achieved; and 3) the changes in ability requirements in response 
to task manipulations exhibited by the different strategy subgroups. In 
Section IV, conclusions and recommendations are presented. 



ERLC 



11 



II. TASK CHARACTERISTICS, PERFORMANCE, AND ABILITIES 



Background 

A recent review (Fleishman & Bartlett, 1969) indicates that laboratory 
studies using combinations of experimental and correlational methods to 
develop principles relating task dimensions to ability requirements are 
rare. Notable exceptions in this regard are studies described by Fleish- 
man (1957) and Zimmerman (1954). Fleishman, for example, attempted to 
relate ability variables to changes in task difficulty represented by systema 
tic alterations of control -display relations in a perceptual -motor task. 
The basic criterion task was a Response Orientation Task, consisting of 
a display panel of 16 lights in circular array and a response panel of 16 
buttons similarly arranged. When a light appeared on the display panel, the 
subject was required to press that button on the response panel which was in 
a specific relative position to the light. Criterion task difficulty was 
manipulated by having subjects perform under eight different degrees of 
display rotation. A factor analysis of criterion data and reference ability 
measures revealed systematic changes in abili4aL:requirements as a function 
of display rotation and consequent task difficulty. For example, under the 
0° condition where the display and response panels corresponded, individual 
differences in performance were primarily a function of the Perceptual Speed 
factor. However, as greater rotations were introduced. Perceptual Speed 
decreased in importance and performance increasingly became a function of 
two other factors— Spatial Orientation and Response Orientation. 

In Zimmerman's study (1954) abilities were investigated as a function 
of changes in the difficulty of a paper-and-pencil perceptual task known as 
Visualization of Maneuvers. In this task the subject was presented with a 
single view of an aircraft as a starting position. An aerial maneuver was 
then described and the subject was to select one of five alternate pictures 
which correctly portrayed the airplane's position following the prescribed 
maneuver. Task difficulty was varied by using three forms of the task 
which required visualization of one, two, or three maneuvers flown in 
sequence. Zimmerman hypothesized that as the visualization task increased 
in difficulty, performance would first be a function of Perceptual Speed, 
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then of Spatial Relations, then Visualization, and finally Reasoning factors, 
in that order. Factor analyses of criterion and ability reference test 
data obtained from large samples of Aviation Cadets tended to support the 
hypothesis for the first three factors. On the easiest and most speeded 
form of the task, performance was a function of the Perceptual Speed factor. 
As task difficulty increased, however, the involvement of this factor 
decreased and the importance of the Spatial Relations and Visualization 
factors increased. 

Considered jointly, the Fleishman and Zimmerman studies relating 
abilities to changes in criterion task difficulty are of fundamental 
importance in understanding the interplay between task characteristics 
and ability requirements. Their value lies in the demonstration that 
manipulations of task difficulty may result in. changes in the patterns of 
abilities accounting for individual differences in performance. Such a 
demonstration is all the more striking when one considers that an alternative 
hypothesis might predict changes in the level of involvement of a specific 
pattern of abilities as a function of changes in task difficulty. Clearly, 
the methodology for relating task characteristics to ability requirements 
must take the possibility of either outcome into consideration. 

Within the present program three additional studies were conducted 
to investigate further the changes, if any, which occur in the patterns 
of abilities accounting for individual differences in performance under 
variations in the criterion task. Since Fleishman and Zimmerman had dealt 
with tasks in the perceptual -motor and visual-perceptual domains, criterion 
tasks were chosen representing more cognitive kinds of performance. These 
included auditory signal identification (Wheaton, Shaffer (Eisner), Mirabel la, & 
Fleishman, 1973), fault finding (Rose, Fingerman, Wheaton, Eisner, & Kramer, 

1974) , and concept identification (Fingerman, Eisner, Rose, Wheaton, & Cohen, 

1975) . 

In each case, the research was conducted in the laboratory using a 
combination of experimental and correlational methods. Based upon an analysis 
of che criterion task, a battery of reference tests was assembled which 
represented abilities judged to be of relevance to criterion task performance. 
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Subjects received the reference battery and then proceeded to perform the 
criterion task under different task conditions. To determine the relation- 
ship between task characteristics and ability requirements, the reference 
battery was factor analyzed to identify a reference ability structure. The 
loadings of the various criterion task conditions on that structure were 
then estimated. The procedures followed in each of the three studies are 
described below together with the major results obtained. 

Auditory-Signal- Identification Task 

Procedure 

In this first study in the program (Wheaton, Shaffer (Eisner), Mira- 
bella, & Fleishman, 1973), a total of 127 male college students received ten 
hours of testing distributed over two consecutive days. Groups of subjects 
were first administered a battery of 24 reference tests which were designed 
to tap into a number of auditory and visual abilities hypothesized as 
potentially relevant to auditory signal identification. Multiple marker 
tests were included for each of six hypothesized abilities to insure adequate 
factor definition. In assembling printed tests to represent the Induction, 
Associative Memory, Speed of Closure, Flexibility of Closure, and Perceptual 
Speed factors, considerable use was made of the Kit of Reference Tests for 
Cognitive Factors (French, Ekstrom, & Price, 1963). Tests selected from 
this kit were used in unmodified form, except that due to time limitations, 
only the first part of all two-part tests was administered. Aural tests 
were based primarily upon Seashore's Measures of Musical Talents (Buros, 
1965) as adapted by Fleishman and Spratte (1954). The major changes made 
were to record standard instructions on tape, together with demonstration 
examples. Other aural tests were taken from among the standardized tests 
of auditory-perceptual abilities developed by Fleishman and Friedman (1957a, 
1957b). 

Upon completion of the reference battery, subjects were exposed to the 
criterion task, which involved the classification of relatively complex 
auditory stimuli into one of four ship categories— submarine, warship, 
cargo, and lightcraft. The stimuli came from a library of ship sounds 
developed by John Annett of the University of Hull. The sounds had been 
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synthesized by electronic and mechanical means to represent realistically 
complex stimuli with the same general characteristics as passive sonar 
signals, but without any real attempt to simulate actual vessels or sonar 
systems. Ship sounds were comprised of several components (i.e., propeller 
cavitation, engine sounds, shaft squeal and hull resonance, sonar, and other 
mechanical ship sounds) which, when combined, constituted four broad cate- 
gories of vessels. Accompanying these signals were background sea noise 
and assorted biological effects. 

The independent variables selected for systematic manipulation of cri- 
terion task difficulty were signal duration and signal-to-noise ratio. Nine 
different task treatment conditions were devised according to a factorial 
arrangement of these two variables. Signals were presented for either 
9-, 6-, or 3-second durations and at one of three signal-to-noise ratios. 
To create the latter, signal strength was kept constant while background 
noise was varied in five decibel (dB) steps— being either weaker (-5 dB), 
stronger (+5 dB), or of the same intensity (0 dB) as the signal. One 
hundred signals, comprised of 25 examples from each of the four categories 
of ship sounds, were presented under each of the nine different task 
conditions. Criterion performance was measured in terms of number of 
correct identifications. 

Results 

Criterion task performance . To assess the impact of manipulating 
the selected independent variables, a fully repeated measures 4x3x3 
factorial design was used in which the performance of all 127 subjects 
was evaluated under the four signal, three duration, and three background 
noise level conditions. The mean percentages of correct identifications 
under these conditions are shown in Figure 1. 

An analysis of variance revealed that performance was significantly 
affected by an interaction between signal category and level of background 
noise. In this interaction, differences in performance associated with 
different signal types decreased as the level of background noise increased. 
Under the lowest level of background noise used, each ship category 
differed from every other category in terms of accuracy of identification 
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(e.g., cargo ships were identified more frequently, and submarines less 
frequently than any of the other ship types), but under the highest 
level of background noise, none of these distinctions was significant. 
Conversely, within each ship category the three levels of background noise 
impacted significantly on performance, identification accuracy systematically 
decreasing with increases in background noise. A similar systematic trend 
was obtained in response to the stimulus duration manipulation. The bulk 
of the effect, however, was confined to the difference between the nine- 
second and shorter intervals. 

Ability structure . Data from the reference battery were intercorrelated 
and factor analyzed by means of a principal components solution. Table 1 
presents the rotated factor loadings obtained using a varimax criterion. 
Of the six major factors extracted, five were tentatively identified, 
including: an Auditory Perceptual factor, a Flexibility of Closure factor, 
an Associative Memory factor, a Speed of Closure factor, and an Induction 
factor. The sixth factor, which had reasonably high loadings on two printed 
measures, could not be readily interpreted. 

Ability requirements as a function of task manipulations . To deter- 
mine the role played by abilities (i.e., factors) in individual differences 
in criterion performance, correlations were computed between the factor 
scores of subjects and their mean levels of criterion task performance. 
Of particular interest were the relationships between the various abilities 
(i.e., factor scores) and performance under the six variants of the 
criterion task attributable to the task characteristic of signal duration 
(i.e., 9, 6, or 3 seconds) and the task characteristic of background noise 
(i.e., -5, 0, or +5 dB). The ui>tained relationships are presented in 
Table 2. Only one of the six factors— Auditory Perception— was an especially 
significant contributor to individual differences in signal identification. 
As the criterion task became more difficult (i.e., when background noise 
increased or signal duration grew shorter), the Auditory Perceptual ability 
showed a corresponding, slight increase in importance. Even more interesting 
was the pattern of correlations between the Auditory Perceptual factor and 
performance under the combinations of ship category and background noise 
conditions which had been found to exert an interactive effect on 

10 



ERIC 



20 



Table 1 
Rotated Factor Matrix 





Reference Tests 


I 




1. 


Letter Sets 


-10 


16 


2. 


Locations 


27 


25 


3. 


Figure Classif . 


13 


13 


4. 


Object-Number 


15 


12 


5. 


Picture-Number 


07 


18 


6. 


First & Last Names 


09 


-04 


7. 


Gestalt Comp. 


17 


10 


8. 


Concealed Words 


13 


12 


9. 


4-Letter Words 


00 


45 


10. 


Concealed Figs. 


18 


63 


11. 


Copying 


04 


75 


12. 


Designs 


02 


82 


13. 


Number Compar. 


-03 


40 


14. 


Identical Pics. 


11 


68 


15. 


Finding A's 


07 


53 


16. 


Pitch 


70 


-13 


17. 


Loudness 


-03 


-12 


18. 


Time 


38 


05 


19. 


Timbre 


58 


18 


20. 


Tonal Memory 


79 


05 


21. 


Rhythm 


67 


15 


22. 


Hidden Tunes 


79 


-04 


23. 


Code Distraction 


57 


07 


24. 


Kwalwasser 


76 


13 



Factors ^ 



III 


IV 


V 




hf 


00 


16 


57 


43 


58 


04 


-16 


58 


08 


50 


-01 


09 


72 


-05 


56 


78 


11 


-01 


07 


66 


82 


-04 


09 


-14 


73 


80 


18 


-01 


17 


71 


29 


62 


40 


-24 


72 


04 


81 


00 


21 


73 


24 


49 


-04 


11 


52 


02 


24 


48 


-08 


72 


06 


05 


22 


-13 


64 


10 


02 


00 


06 


69 


07 


05 


-18 


60 


56 


-10 


14 


31 


01 


59 


13 


02 


03 


17 


34 


06 


17 


23 


-10 


60 


-05 


11 


10 


72 


55 


18 


-09 


17 


54 


51 


07 


05 


-07 


27 


46 


-07 


18 


06 


-12 


68 


01 


-21 


-13 


04 


54 


15 


05 


18 


05 


69 


17 


02 


45 


20 


j60 


14 


08 


15 


-05 


64 



^ Factor loadings reflected and rounded to two places; decimals omitted. 

Factors are defined as: I - Auditory Perceptual; II - Flexibility of Closure; 

III - Associative Memory; IV ~ Speed of Closure; 
V - Inductive Reasoning; VI - Undefined. 

(From Wheaton et al., 1973.) 

11 



21 



Table 2 



Correlations Between Factor Scores and Performance 
Under Variations of the Criterion Task ^ 

FactOY^s 



Criterion Task 





Variation 


I 


H. 


III 


IV 


V. 


VI_ 


h! 


R 


1. 


9" 


35 


00 


-01 


08 


11 


13 


16 


40 


2. 


6" 


36 


-01 


07 


02 


18 


11 


18 


42 


3. 


3" 


39 


-04 


06 


02 


12 


08 


18 


43 


4. 


-5dB 


33 


-05 


06 


06 


15 


11 


15 


39 


5. 


OdB 


36 


00 


04 


06 


15 


11 


17 


41 


6. 


+5dB 


41 


02 


00 


00 


09 


10 


19 


43 



Correlations rounded to two places; decimals omitted. 

Factors are defined as: I - Auditory Perceptual; II - Flexibility of Closure; 

III - Associative Memory; IV - Speed of Closure; 

V - Inductive Reasoning; VI - Undefined. 

(Adapted from Wheaton, et al., 1973.) 
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identification accuracy. As shown in Figure 2, the correlations increased 
across background noise conditions for two of the ship categories and 
decreased for the other two kinds of signals. 

Implications 

With respect to the general methodological issues addressed by the 
study, the crux of the results lay in the correlations between performance 
under the different criterion task conditions and individuals' factor 
scores. Generally speaking, changes in these correlations occurred as a 
function of the signal duration and signal-to-noise ratio manipulations. 
As either of these task characteristics was varied so as to increase 
task difficulty, the correlations with one set of factor scores changed 
accordingly. 

The nature of the change in ability requirements differed from that 
found in the studies by Fleishman (1957) and Zimmerman (1954). In both 
these earlier studies specific abilities involved in criterion task 
performance dropped out while others came into play as a function of systematic 
variations in task characteristics; that is to say, the abilities contributing 
to performance under different versions of the task changed. In the present 
study, a single ability was involved under all task conditions, its .importance 
waxing or waning slightly as a function of variations in task characteristics. 

It can be argued, of course, that differential involvement of abilities 
in task performance is not a necessary outcome of certain kinds of task 
variations. As long as the same basic task is being performed, the same 
pattern of abilities can be relevant. Variations in the conditions under 
which that task is performed, therefore, while contributing to task 
difficulty, may affect only the degree to which the relevant pattern is 
involved and not the pattern itself. This suggests that it may be necessary 
to differentiate among two kinds of variables affecting task difficulty. 
For example, varying certain kinds of variables (as in the Fleishman and 
Zimmerman studies) may cause subtle changes in the basic task itself, 
resulting in changes in the pattern of abilities related to different 
versions of the task. Varying other kinds of variables (such as signal 
duration or signal-to-noise ratio in the present study) may have no 
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Figure 2. Estimated correlations between performance on the four signal 
categories and Auditory Perceptual ability as a function of 
background noise level. 



(From Wheaton et al., 1973.) 
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effect on the basic nature of the task to be performed; in this case, 

the importance of a fixed pattern of abilities would be expected to increase 

or decrease as the conditions of performance became more or less demanding. 

To address the issue of change-in-pattern versus change-in-degree of 
ability involvement a second study was conducted in which the ability 
requirements related to different versions of an electronic fault-finding 
task were investigated. This type of task was chosen because of its a 
priori potential for exhibiting either kind of outcome. 

Electronic Fault-Finding Task 

Procedure 

The general procedure employed in this study was again to administer 
a battery of reference ability tests and several variations of a single 
criterion task to a large number of subjects (Rose, Fingerman, Wheaton, 
Eisner, & Kramer, 1974). To determine the relationships between task 
characteristics and ability requirements, the reference battery was factor 
analyzed and performance scores obtained under the various criterion task 
conditions were correlated with subjects* factor scores. The resultant 
coefficients were examined to determine whether there were any systematic 
relationships between changes in task dimensions and the patterns of 
abilities contributing to individual differences in performance. 

An electronic fault-finding task (representative of situations faced 
by electronic troubleshooters) was employed as the criterion task. It 
consisted of a series of problems in which subjects were required to 
locate breakpoints in hypothetical electronic devices. The basic format 
was a current-flow diagram or digital -logic circuit in which the state of 
the output at any point was determined by the preceding logic gates. In 
each circuit a single faulty wire, or "breakpoint" was introduced. At such 
a point the current flow was disrupted. The subject's task was to identify 
the location of this break by probing the circuit at various locations 
while depressing different combinations of switches. The troubleshooting 
task was systematically varied along two dimensions. First, formal diffi- 
culty was manipulated by increasing the number of possible breakpoints 
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and the number of logic gates in the circuit. The second manipulation was 
to vary the perceptual complexity of the problem by changing the configura- 
tion or layout of the circuit. Circuit diagrams were created such that 
different circuits were topologically equivalent, but their spatial appear- 
ances were quite varied. 

Three levels of each dimension were used; thus, nine different test 
problems were generated. Each problem was presented twice, with different 
breakpoints as solutions, providing a total of eighteen experimental prob- 
lems. Several dependent measures were examined, including time to solu- 
tion, number of trials (probes) to solution, time per trial, efficiency 
(expressed in terms of the proportion of breakpoints which were eliminated 
by a probe relative to the maximum number which could have been eliminated 
on that test) on the first probe, and overall efficiency. 

The reference test battery consisted of 21 tests representing six 
well-established factors in the cognitive, perceptual, and memorial domains 
of performance. The specific factors chosen were hypothesized to be 
relevant to criterion task performance. To insure adequate factor defini- 
tion, each of these factors was represented by a minimum of three tests. 
A total of 135 subjects were administered the criterion problems and the 
reference battery. 

Results 

Criterion task performance . Analyses of variance conducted on the 
criterion task data revealed that while both task manipulations (formal 
difficulty and perceptual complexity) affected task performance, they did 
not necessarily affect the same aspects of performance (i.e., dependent 
measure), nor did they affect particular aspects in the same way. For 
example, time-to-solution and time-per-trial were both influenced strongly 
by both formal difficulty and perceptual complexity. Illustrative results 
are shown in Figure 3 for the time-to-solution measure. Trial-one 
efficiency was affected to a lesser degree by perceptual complexity, and 
almost not at all by formal difficulty. Trials-to-solution and efficiency- 
per- trial were more sensitive to changes in formal difficulty than to 
changes in perceptual complexity. 
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Figure 3. Geometric mean time to solution as a function 
of formal difficulty and perceptual complexity. 
(From Rose et al . , 1974.) 
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This interplay between task manipulations and performance measures 
was quite important, since the different measures might be related to differ- 
ent sets of abilities. If this were the case, the relationship between 
task characteristics and ability patterns might depend upon the performance 
measures under consideration. Thus, different regression equations would 
be necessary to predict different aspects of criterion performance from 
abilities, over and above any changes in predictive equations made necessary 
by manipulations of the criterion task itself. 

Ability structure . Five major factors were extracted from the inter- 
correlations among reference tests using a principal components solution. 
Orthogonal rotation of the factors was performed using a varimax criterion. 
Factors were interpreted for psychological meaningful ness from projections 
of the reference tests on the rotated axes as presented in Table 3. The 
five factors were as follows: Flexibility of Closure/Spatial Scanning, 
Syllogistic Reasoning, Associative Memory, Perceptual/Cognitive Speed, 
and Induction. 

Ability requirements as a function of task manipulations . Relation- 
ships between ability requirements and task characteristics were again 
explored by correlating performance under the various task manipulations 
with subjects* levels of ability as reflected in their factor scores. 
As shown in Table 4, from 10% to 38% of the variation in the dependent 
measures could be accounted for by the reference factors. This 
common variance was primarily associated with Factors I (Flexibility 
of Closure/Spatial Scanning), II (Syllogistic Reasoning), III (Associative 
Memory), and V (Induction). The relationships between criterion perfor- 
mance and abilities (i.e., changes in coefficients as a function of levels 
of formal difficulty and perceptual complexity) were examined for each 
performance measure. 

The general conclusion was that different abilities were involved, and 
at different levels of involvement, when either the task dimensions varied 
or different dependent measures were examined. It was found that the 
loadings on Factor I generally tended to increase as both formal difficulty 
and perceptual complexity increased. This relationship held across most 
performance measures. Factor II was moderately involved in all but the 
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Table 3 



Factor Loadings ^ in Rotated Factor Matrix 





m 

Reference Tests 


Factors 
I II 


T T T 

III 


Iv 


V 


.2 
h 


1 . 


Granmatical Reasoning 


01 


66 


18 


39 


10 


63 


2. 


Maze Tracing Speed 


69 


-01 


-01 


32 


00 


58 


3. 


Copying 


71 


08 


-07 


30 


30 


70 


4. 


Logical Reasoning 


27 


63 


18 


-08 


27 


59 


5. 


Letter Sets 


18 


28 


15 


15 


52 


42 


6. 


Identical Pictures 


28 


13 


09 


62 


10 


49 


7. 


First and Last Names 


-07 


25 


77 


10 


-08 


68 


8. 


Locations 


15 


05 


14 


27 


62 


50 


9. 


Object-Number 


10 


06 


85 


10 


12 


77 


10. 


Finding A's 


33 


11 


12 


49 


-10 


39 


11. 


Search (mean) 


-10 


-26 


-04 


-77 


04 


67 


12. 


Number Comparison 


05 


-11 


06 


67 


32 


57 


1 J . 


Desi gns 


59 


-01 


00 


0 1 




/ u 


14. 


Nonsense Syllogisms 


00 


78 


08 


00 


08 


62 


15. 


Closure Flexibility 


70 


24 


00 


37 


21 


72 


16. 


Choosing a Path 


74 


17 


05 


-20 


06 


63 


17. 


Inference 


05 


71 


02 


18 


06 


54 


18. 


Map Planning 


53 


11 


07 


38 


30 


54 


19. 


Permutations 


08 


21 


02 


-13 


69 


55 


20. 


Picture- Number 


07 


07 


83 


08 


24 


77 


21. 


Figure Classification 


40 


61 


09 


01 


23 


60 



^ Factor loadings reflected and rounded to two places; decimals omitted. 



Factors are tentatively defined as: 

I - Flexibility of Closure/Spatial Scanning 
II - Syllogistic Reasoning 
III - Associative Memory 
IV - Perceptual /Cognitive Speed 

V - Induction 

(From Rose et aK, 1974.) 
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Table 4 

Correlations Between Factor Scores and Perftfrmance 
Under Variations of the Troubleshooting Task ^ 



Criterion Measures 



Factors 
I II 



III 



IV 



R 



Trial 1 Efficiency 



1 
2 
3 
A 
B 
C 



27 
26 
33 
19 
32 
35 



18 
21 
19 
20 
25 
15 



16 
05 
21 
13 
13 
17 



09 
06 
03 
01 
03 
14 



10 
10 
07 
17 
09 
04 



15 
13 
19 
12 
19 

19 



38 
35 
44 
35 
43 
44 



Ln Trials to Solution 



Arcsin Square Root 
Efficiency per Trial 



n Time to Solution 



Ln Time per Trial 



1 

2 
3 
A 
B 
C 

1 

2 
3 
A 
B 
C 

1 
2 
3 
A 
B 
C 

1 
2 
3 
A 
B 
C 



15 
30 
34 
19 
27 
34 

30 
33 
50 
34 
38 
42 

25 
25 
32 
19 
27 
41 

16 
10 
08 
10 
10 
13 



20 
22 
12 
19 
17 
20 

30 
35 
31 
31 
35 
31 

14 
14 
10 
11 
13 
17 

-01 
03 
02 
01 
03 
01 



09 
09 
04 
12 
04 
07 

20 
19 

12 
30 

18 
05 

05 
-04 
-08 
-04 
-01 
-04 

-02 
-09 
-11 
-12 
-03 
-09 



-09 
-08 
-00 
-08 
-13 
03 

-04 
-02 
01 
01 
-04 
-02 

-06 
-04 
-04 
-05 
-09 
00 

00 
00 

-04 

-01 

-01 

-02 



17 
15 

10 
30 
08 
07 

12 
23 
14 
28 
11 
14 

29 
23 
29 
31 
23 
29 

18 
17 
21 
17 
18 
22 



10 
17 
14 
19 
13 
17 

23 
32 
38 
38 
31 
30 

18 
13 
21 
15 
15 
28 

06 
05 
07 
05 
05 
07 



Signs have been reflected to relate superior performance to 
superior ability; decimals omitted. 

Factors are tentatively defined as: 



I 

II 
III 
IV 
V 



Flexibility of Closure/Spatial Scanning 
Syllogistic Reasoning 
Associative Memory 
Perceptual/Cognitive Speed 
Induction 



(From Rose et al 1974.) 



33 
42 
38 
43 
36 
41 

48 
57 
62 
61 
56 
54 

42 
37 
45 
38 
38 
53 

24 
22 
26 
23 
21 
27 
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time measures. Further, its level of involvement was fairly constant, i.e., 
it did not change as a function of formal difficulty or perceptual complexity. 
Next, the loadings on Factor V tended to decrease as a function of increasing 
perceptual complexity on the trial s-to-solution and efficiency-per-trial 
measures. On the remaining measures, loadings for this same factor were 
moderate but constant. Finally, Factor III loadings decreased as a function 
of perceptual complexity for the efficiency-per-trial measure. 

Implications 

Given the nature of these results, one implication was clear. If one 
were to use ability criteria to predict who would do well on these problems, 
the choices would differ depending on the task characteristics and the 
criterion measure selected. There were, in fact, two general types of 
selection decisions implied by the data. One type of decision occurred 
when a change in task demand (due either to different criterion measures 
or changes within a task dimension) did not involve different abilities. 
In this case, a change in cut-off values would be the appropriate decision. 
The abilities involved were the same, but they were involved to a greater or 
lesser degree as the task demands changed. The second type of decision 
occurred when the task demands were altered and a different set of abilities 
was involved.. In this type of situation, since different abilities (or 
factors) were relevant, the implication was that entirely different selection 
criteria would have to be given consideration. 

It would be desirable if one kind of task manipulation (or one aspect 
of performance) consistently implied one or the other type of decision. 
However, there was no consistent relationship discernable between type of 
manipulation (or measure) and pattern of ability requirement. It was 
felt, however, that such consistent relationships might emerge were more 
control exercised over the possible strategies subjects might utilize. 
The problem is that any change in a given task characteristic might be 
perceived as a new problem by some subjects, but as the same problem by 
others. To the extent that these perceptions lead the first group to 
adopt alternative strategies for the "new" problems, different abilities 
may be involved. The second group, while maintaining their strategy, 
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would utilize the same abilities, but perhaps at different levels of 
involvement. An additional study was conducted, therefore, to shed light 
on these issues. 

Concept-Identification Task 

Procedure 

In the third data collection effort (Fingerman, Eisner, Rose, Wheaton, & 
Cohen, 1975) the relationship between variations in a prototypic problem- 
solving task, concept identification, and consequent changes in the abilities 
related to problem-solving performance was investigated. A total of 128 
college students participated in the experiment, which was administered 
in two parts— a concept-identification criterion task and a battery of 21 
paper-and-pencil reference tests representing seven well-established factors 
in the cognitive, perceptual, and memorial domains of performance. As in 
the earlier studies, the specific factors chosen were hypothesized to be 
relevant to criterion task performance. 

The simultaneous concept-identification paradigm was used for the cri- 
terion task, with the stimuli consisting of pairs of human faces. On each 
problem one facial characteristic (e.g., eyes) and one attribute of that 
feature (e.g., "squinty") were preselected by the experimenters as the solu- 
tion. The subject's task was to identify that one correct facial charac- 
teristic after five consecutive trials, during which pairs of faces were 
projected on a screen in front of the room. For each pair of faces (or 
trial), the subject indicated his choice of left-hand or right-hand face, 
was told which face in the pair contained the solution, and was asked to 
indicate (on a list of all possible solutions to the problem) those facial 
characteristics he thought could still be the solution to the problem. 
The number of characteristics indicated was considered to be the number of 
hypotheses being entertained by or tested by the subject at that particular 
time. A new pair of faces was then presented to begin the next trial. 
Following the fifth trial, he was asked to write down the one single charac- 
teristic which he thought was the solution to the problem. 

Two characteristics of the task were manipulated: (1) the number of 
stimulus dimensions or facial characteristics (formal difficulty), and 
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(2) the extent to which location of facial features varied from normal 
appearance (perceptual complexity). Three levels of each of the manipu- 
lations were created, and subjects were tested under all versions of the 
criterion task in a three-by-three, within-subjects design. 

Ten problems were presented at each level of difficulty, with the 
first problem in each group always being a sample problem. Of the remain- 
ing nine problems at each level of difficulty, there were three problems 
at each level of perceptual complexity. Subjects were presented first 
with the set of four-dimensional problems, then with the six-dimensional 
problems, and finally with the eight-dimensional problems. 

Results 

Criterion task performance . A variety of measures of performance 
on the concept-identification task were available. Three representative 
measures were selected for use in the detailed analyses, including: (1) 
proportion of problems solved under each task condition; (2) an efficiency 
measure reflecting the number of attributes tested on each trial relative 
to the optimal number possible (A/B); and (3) a measure reflecting the 
efficiency of the set of attributes selected for testing (C^/AB). Analyses 
of variance revealed strong impacts on performance due to the two task 
characteristics which were manipulated. 

Each of the dependent measures was influenced by the stimulus-dimension 
variation. With respect to the proportion of problems solved, the four- 
dimension problems were solved significantly more often than the pooled 
six- and eight-dimension problems, while there was no significant difference 
between six- and eight-dimension problems. Similarly, with respect to 
either efficiency variable, subjects were significantly more efficient when 
solving the four-dimension problems than when solving six- or eight-dimension 
problems. Thus, increasing the number of dimensions from four to s^x or 
eight impaired performance across all three measures. 

The impact of perceptual complexity was more complicated. Perceptual 
complexity did not exert any directly interpretable influence on the pro- 
portion of problems solved. However, increasing the perceptual complexity 
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did result in different effects for the two efficiency measures. Added 
complexity, resulted in a significant increase in processing efficiency as 
represented by the C^/AB measure. This improvement is consistent with 
empirical findings (Postman. 1972; Tulving & Donaldson, 1972) which suggest 
that better recall performance is produced when the normal face presentation 
is altered, thereby requiring the subject to actively organize the set of 
facial features, rather than passively use the "natural" top-to-bottom order 
found in problems at the first level of perceptual complexity. Obviously, 
with improved recall, the subject is in a better position to use efficiently 
feedback information to narrow down the set of hypotheses he is considering. 
The effect on the A/B efficiency score, however, was moderated by level of 
problem difficulty": The interaction of difficulty and complexity on the 
A/B measure (subjects tested a more optimal number of hypotheses with 
increased complexity on easy problems, but tested less optimally as com- 
plexity increased on difficult problems), suggests that the strategy or 
approach adopted by a subject is dependent on the total cognitive load of 
the task. In the four-dimension problems, where the total number of 
hypotheses was small, subjects were able to handle the additional load 
imposed by increased perceptual complexity. In the eight-dimension problems, 
where more total possible solutions were involved, an increase in perceptual 
complexity required the subject to actively construct his own list organi- 
zation as well as scan the increased number of relevant facial characteristics. 
Apparently this additional burden caused the subject to reduce his net 
cognitive load by decreasing the number of hypothesized solutions which he 
tested at any one time. 

Ability structure. Six factors were extracted from the intercorrela- 
tions among reference tests using a principal components solution. Orthogonal 
rotation of the factors was performed using a varimax criterion. The 
rotated matrix of factor loadings is presented in Table 5. The factors 
were tentatively identified as: I - Flexibility of Closure/Spatial Scanning; 
II - Associative Memory; III - Perceptual Speed; IV - Syllogistic Reasoning; 
V - Speed of Closure; and VI - Induction. 

Ability requirements as a function of task manipulations . Correlations 
between performance, under the difficulty (4D, 6D, 8D) and complexity (A.B.C) 
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Table 5 



Factor Loadings ^ in Rotated Factor Matrix 





Reference Tests 


I 


T T 

II 


Factors 
III IV 


V 


VI 


u2 
X\'- 


1. 


Grammatical Reasoning 


16 


20 


1 0 


10 


17 


24 


69 


2. 


Neisser Search 


22 


05 


/ H 




-03 


01 


59 


3. 


Picture Number 


09 


79 


1 D 




-04 


02 


69 


4. 


Maze Tracing Speed 


71 


-03 


1 / 


Ml 


19 


18 


60 


5. 


Inference 


15 


14 


-21 


11 


-06 


12 


62 


6. 


Locations 


34 


02 


-14 


06 


14 


66 


59 


7. 


First and Last Names 


-08 


79 


07 


-01 


14 


14 


67 


8. 


Letter Sets 


20 


14 


26 


23 


-02 


68 


64 


9. 


Concealed Words 


16 


10 


22 


05 


82 


-02 


75 


1 n 


P prmi 1 f* A f* 1 n n Q 
r CI iiiu i/u Kf 1 V 1 1 o 


— U 1 


CO 


23 


24 


-Ml 


ou 




11 . 


Gestalt Completion 


30 


-00 


-18 


14 


74 


14 


71 


12. 


Four-Letter Words 


08 


06 


64 


-13 


32 


17 


58 


13. 


Map Planning 


55 


06 


02 


35 


01 


39 


58 


14. 


Designs 


71 


20 


09 


28 


09 


05 


65 


15. 


Copying 


78 


15 


15 


08 


16 


22 


73 


16. 


Identical Pictures 


75 


-03 


31 


-07 


09 


-06 


67 


17. 


Choosing a Path 


46 


-01 


-24 


17 


20 


53 


62 


18. 


Nonsense Syllogisms 


12 


-19 


02 


72 


15 


06 


59 


19. 


Number Comparison 


35 


17 


63 


05 


-02 


-06 


56 


20. 


Closure Flexibility 


69 


15 


-02 


27 


26 


37 


78 


21. 


Object Number 


24 


83 


-06 


-04 


03 


03 


75 



^ Factor loadings reflected and rounded to two places; decimals omitted. 

Factors are tentatively defined as: 

I - Flexibility of Closure/Spatial Scanning 

II - Associative Memory 

III - Perceptual Speed 

IV - Syllogistic Reasoning 

V - Speed of Closure 

VI - Induction 

(From Fingerman et al*» 1975.) 
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conditions, and factor scores are shown in Table 6. From 15% to 45% of 
the variance in criterion task performance could be accounted for by 
individual differences on the six factor-defined abilities. Since the 
coefficients for the Perceptual Speed and Speed of Closure factors were 
uniformly low, most of the variance was accounted for by the Flexibility 
of Closure/Spatial Scanning, Associative Memory, Syllogistic Reasoning, 
and Induction factors. These four factors all showed moderate to fairly 
strong involvement which varied as a complex function of the task manipu- 
lations and the dependent variable considered. For example, as shown in 
Figure 4, the correlations between Associative Memory and proportion of 
problems solved varied as problem difficulty increased but remained rela- 
tively constant across levels of perceptual complexity. In a similar vein, 
correlations with the Flexibility of Closure/Spatial Scanning ability 
showed an initial increase and then a marked decrease as the number of 
dimensions increased; correlations with this same ability increased across 
levels of perceptual complexity. 

Implications 

Results from this study showed that as task characteristics were varied, 
systematic changes in performance did occur, and the abilities contributing 
to performance also changed. However, covariation of abilities with performance ' 
did not seem to be a simple function of task manipulations. This suggested ^ 
that individual subjects may have utilized different approaches as a function 
of how they perceived the task to have changed. From a cognitive point of 
view, the way subjects view the task determines the way in which they will 
attempt to deal with it, and this in turn may determine which abilities are 
called into play in explaining individual differences in performance. If a 
task variation does not change the nature of the task from the subjects' 
point of view, then their approach should remain the same and there would 
be no change in the pattern of abilities contributing to individual differ- 
ences in performance. If the task is viewed as a new one, then the method 
of attacking it may change and the abilities contributing to performance 
may be completely different. 
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• Table 6 

Correlations Between Factor Scores and Performance Under 
Variations of the Concept Identification Task ^ 

Factors 



Criterion Variabl 


as 


I 


TI 


III 


IV 


V 


VI 




Proportion Solved: 
















PercpDtual fnm- 


















plexity 


A 


.14 


.33 


.15 


.29 


.11 


.17 


.27 




B 


.27 


.34 


.00 


.20 


.12 


.25 


.30 




C 


.30 


.33 


-.01 


.31 


.06 


.20 


.33 


Dimensions 


4D 


.23 


.27 


.13 


.21 


.17 


.16 


.24 




6D 


.32 


.37 


.00 


.27 


.01 


.16 


.34 




8D 


.15 


.32 


.02 


.27 


.11 


.26 


.28 


A/B : 


















Perceptual Com- 


















plexity 


A 


.25 


.28 


.15 


.36 


.10 


.25 


.37 




B 


.31 


.26 


.14 


.35 


.08 


.28 


.39 




C 


.26 


.28 


.15 


.36 


.09 


.27 


.38 


Dimensions 


4D 


.24 


.15 


.23 


.36 


.02 


.28 


.34 




6D 


.29 


.27 


.09 


.38 


.10 


.25 


.38 




8D 


.26 


.34 


.11 


.28 


.13 


.24 


.35 


C^/AB : 


















Perceptual Com- 


















plexity 


A 


.17 


.37 


.15 


.38 


.14 


.24 


.41 




B 


.27 


.39 


.07 


.32 


.09 


.30 


.43 




C 


.28 


.35 


.04 


.35 


.10 


.26 


.40 


Dimensions 


4D 


.22 


.29 


.15 


.35 


.09 


.27 


.36 




6D 


.30 


.40 


.04 


.35 


.10 


.24 


.45 




8D 


.18 


.36 


.07 


.31 


.12 


.26 


.35 



^ Signs have been reflected to relate superior performance to superior 
ability; factors are identified as: I - Flexibility of Closure/Spatial 
Scanning; II - Associative Memory; III - Perceptual Speed; IV - Syllogistic 
Reasoning; V - Speed of Closure; VI - Induction. 

"(From Fingerman et al., 1975.) 
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While the particular kinds of approaches used by subjects in concept- 
identification tasks are, perhaps, of limited generalizability, the implica- 
tions of the research are very important. If the linkage of change in task 
characteristic, change in strategy, and change in individual differences 
can be validated, it should become possible in the short term to predict 
whether-fi new version of an old task will require revision of personnel 
selection or training criteria. If job incumbents were not expected to 
change their approach (based on empirical or theoretical analysis of the 
tasks involved), no new selection instruments or training courses would 
be required (although performance criteria in selection or training might 
require change). If a change in approach were predicted to occur, one 
would then need to determine the new ability requirements after ascertaining 
which new approach had been adopted. 

The next section of this r^eport describes how the concept-identification 
and fault-finding studies were reevaluated in an attempt to address the 
issue of an interaction among task variations, subject strategies, and 
ability requirements. 
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III. TASK CHARACTERISTICS, STRATEGIES, AND ABILITIES 

Introduction 

In order to examine the hypothesized mediating role of strategies 
between task characteristics and abilities, it was necessary to develop 
a classification scheme for the different strategies used by subjects 
while performing on the various versions of the two cognitively-oriented 
criterion tasks. Fortunately, a great deal of theoretical and empirical 
work has been done on strategy analysis in concept-identification tasks, 
and the same concepts appeared applicable to the fault-finding task 
as well. Levine (1966, 1970) has characterized subject behavior in 
concept-identification problems as a series of hypothesis-testing operations. 
On each trial the subject chooses one or more attributes which he believes 
(hypothesizes) may be the solution to the problem, and attempts to test 
whether or not the solution is, in fact, the attribute (or one of the 
attributes) he has chosen. Due to the nature of the problem, he is able 
on each trial to eliminate or confirm part of his "working set" of attributes. 
He continues in this manner to the next trial with those members of the 
set which have been confirmed or have not yet been eliminated. 

Bruner, Goodnow, and Austin (1956) distinguished two major forms 
of strategies in concept identification which may be cast into Levine 's 
model as follows: 

1. The wholist strategy, in which a subject takes as his working 
set prior to the first trial all possible attributes. Based on feedback 
from each trial he eliminates attributes until he has narrowed his set 
down to the single, correct attribute. 

2. The part-scanning or £art strategy, in which a subject takes 

as his working set prior to the first trial a subset of all the possible 
attributes. Based on feedback from each trial he eliminates attributes 
until he reaches the solution, provided the solution was originally contained 
in the working set, or starts over with a second sample of attributes as 
his new working set, from which he continues to eliminate attributes. 

The wholist strategy has also been referred to as focusing, and the 
part strategy as subset sampling. Levine (1975) has presented an extensive 
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model of these strategies which allowed us to develop a scheme for 
identifying the strategy used by a subject in attempting to solve 
problems posed by the concept-identification and fault-finding tasks. 

The purpose in attempting to identify the strategies utilized by sub- 
jects was to determine their role in mediating the task-characteristic 
ability interaction. The mediational situation might be characterized 
as a mul ti-variate model which predicts performance from three variables 
and their interactions: task characteristics, ability profile, and 
strategy. The effects of the first two variables have been reported upon 
in the preceding chapter. In order to determine the usefulness of strategy 
in this system, one must look at the effect of strategy on performance 
(e.g., the "main effect" of the intervening variable), the interaction of 
strategy and task characteristics, the interaction of strategies and 
abilities, and finally the second-order interaction which examines the 
ability-task relationship for various strategies. 

The extent to which groups of subjects with certain ability patterns 
have predictable strategy patterns represents the strategy-ability inter- 
action. For example, one may ask whether subjects who focus under all task 
conditions have different ability patterns from subjects who test subsets 
under all conditions, and whether these two groups differ from those subjects 
who focus on easy problems, and turn to subset testing when the problems 
become more difficult. 

The crucial mediating role of strategies concerns the extent to which 
the task-characteristic ability relationships are clarified when sets 
of subjects with more or less homogeneous strategies are examined, 
i.e., the strategy-ability-task interaction. Thus, while the task- 
characteristic ability correlations previously reported were highly complex 
when examined for all subjects combined, it might be that these patterns 
may be simpler to interpret when based on subjects who adopt a particular 
strategy. 

While no thorough analysis of the abilities involved in applying 
each strategy under different task conditions has previously been performed, 
several ability-strategy relationships are strongly implied by existing 
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iDodels. For example, consider the subset-tester: in the unlucky event that 
his initial working set does not contain the solution to the problem, he 
must attempt to form a new set which overlaps as little as possible with 
the original one, i.e., which contains new attributes which he has not yet 
eliminated. This construction of a new and non-overlapping set is successful 
to the extent that he can remember the attributes previously eliminated 
(not a requirement if the original set contains the solution). Thus, 
the^subsetter's performance depends to some extent on memory. The focuser's 
task, on the other hand, is one of attribute elimination; therefore, his 
memory ability should not be strongly related to performance. 

Procedure 

The first step in determining the role of strategies was to extract 
the "strategy" information from subjects' performance protocols. The 
method chosen was to develop criteria for the assignment of a strategy 
classification label to every problem attempted by each subject in both 
the concept-identification and troubleshooting tasks. Since strategies 
in the concept-identification paradigm can be unambiguously defined, labels 
and criteria for their assignment to protocols in that task were generated 
first; analogous labels and criteria were then derived for the troubleshooting 
task. As described in the previous report in this series (Fingerman et al., 
1975), subjects were required to report which attribute(s) of the stimuli 
they thought were still solution possibilities after each trial. Since 
the stimulus sequences were orthogonal, it was possible for a subject to 
eliminate half of the remaining possible solutions on each trial, if a per- 
fect focusing strategy were adopted. With respect to the protocol, this 
perfect focusing would appear as follows: 

1. F 'lowing the first trial, exactly half of the attributes would 
be checked as possible solutions. 

2. The number of attributes checked as possible solutions would 
decrease by half after each trial until the correct solution was arrived at 
(e.g., in the 8-dimension problems, the number of attributes checked would 
proceed from 8 to 4 to 2 to 1 over the first four trials), 
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3. Each attribute checked would be both locally consistent (consistent 
with feedback information from the immediately preceding stimulus presen- 
tation) and globally consistent (consistent with respect to all preceding 
trials, Gregg & Simon, 1967). 

If these three conditions were met on any given problem protocol, it 
was labeled as "Excellent Focusing." However, it was possible for a 
subject to deviate from this ideal pattern while still attempting a focusing 
strategy (and, incidentally, still solving the problem). For example, a subject 
could simply "lose" a stimulus attribute during the course of an 8-dimension 
problem by forgetting one of the members of his original working set. 
This and other errors would produce some deviations in the problem protocols. 
To allow for these cases, a "Focusing with Errors" label was created by 
relaxing some of the above criteria as follows: 

1. A full working set was still required after the first trial; 

2. The number of attributes checked as possible solutions would 
decrease after each trial, while allowing for deviations from an orthogonal 
reduction (e.g., an acceptable 8-dimension protocol would be 8 to 3 to 2 to 1 ) ; 

3. The locally consistent and globally consistent requirements were 
relaxed; most, but not all, checked attributes were required to be con- 
sistent with preceding feedback information. 

The third strategy classification label, "Subsetting," was 
applied to protocols using the following rules: 

1. Subjects took as their initial working set a number of attributes 
less than the maximum possible (e.g., fewer than eight attributes on an 8- 
dimension problem). 

2. The working set was systematically reduced until the solution 
was arrived at, or in cases where the solution was not contained in the 
working set, a new subset was selected. 

3. Each attribute checked was to be both locally and globally con- 
sistent with previous information. 

Again, these criteria were modified to allow for errors, while 
still falling under the "subset" category; the "locally and globally 
consistent" requirement was relaxed, as was the "systematic reduction" 
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criterion, yielding a "Subsetting with Errors" label. 

The experimental procedures employed in the concept-identification 
task (i.e., having subjects list their trial-by-trial hypotheses) was an 
attempt to make overt the normally covert reasoning processes used by 
subjects. For the troubleshooting task, however, these reasoning processes 
could only be inferred from a more limited protocol: the ordered list 
of tests selected (Rose et al., 1974). It was assumed that the breakpoints 
actually included in each test represented the subject's working set of 
possible solutions. Given this assumption, strategy classifications similar 
to those made in the concept-identification task were made from the inferred 
working set of breakpoints on each trial. For example, a "split-half" 
technique in the troubleshooting task (wherein exactly half of the possible 
breakpoints are eliminated on each trial) is logically equivalent to a 
focusing strategy. Similarly, the troubleshooting technique of "working 
forward" (wherein breakpoints are eliminated in a logical order by starting 
with those closest to a given switch and working outward toward the terminals) 
is logically equivalent to subsetting. 

Thus, the initial criterion for classification of protocols in the 
troubleshooting task as either focusing or subsetting was whether subjects 
attempted a "split-half" or "working-forward" approach. Through an iterative 
process, it was possible to develop criteria in order to distinguish between 
excellent, good, and poor focusing strategies and subset strategies for 
each protocol. In order to qualify as an excellent focusing protocol, 
the following rules were adopted: 

1. The initial test had to include a terminal probe point. 

2. All succeeding tests approximated a split-half technique (since, 
for the problems employed, tests of exactly half of the remaining alterna- 
tives were frequently impossible). 

3. No tests were completely redundant or completely ambiguous. 

The criteria for a good focusing protocol were: 

1. The initial test had to include a terminal probe point. 

2. All tests after the first non-erroneous negative feedback (i.e., 
when the probe light did not light because the breakpoint was contained 
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in the probe path) approximated a split-half technique. 

3. Most tests (for the entire protocol) tested more than half of the 
breakpoints that could logically be eliminated (e.g., if it were possible 
to test eight potential breakpoints, four or more were tested). 

Protocols wherein the first probe included a terminal point but did not 
meet the remainder of the criteria for good focusing were classified as 
poor focusing. Formal criteria for classification of subset protocols were 
developed; however, upon reviewing the protocols it became apparent that 
subsetting protocols were easily distinguishable from all others. The only 
criterion that was necessary was that the initial test was not at a terminal 
point. In all cases where protocols were classified as subsetting, it was 
clear that subjects had chosen not to attempt to test a full working set, 
but rather had chosen a particular subset of possible breakpoints to sample. 

Given these criteria, project staff proceeded to classify each problem 
protocol for both studies. Ambiguous classifications were resolved by 
consensus. Once these basic data were generated, analyses were conceptualized 
to address the ability-strategy-task characteristic issues discussed above. 

Results 

Strategies and Performance 

The first series of questions addressed the extent to which there were 
patterns of strategy [Utilization as a function of task manipulations across 
subjects. Also of interest was the degree to which there were any consis- 
tencies in the way individual subjects employed strategies. 

Tables 7 and 8 show the number of subjects using each strategy as a 
function of the task characteristics for each of the two studies. Notice 
that the left margin headings, labeled Excellent, Good, or Poor Focuser, 
Subsetter, and Inconsistent, are designations of subgroups of subjects . 
After all problem protocols had been categorized, each subject was classified 
into one of the above subgroups for each task-characteristic marginal, as 
well as being assigned an overall subgroup label. On the concept-identificatii 
task (Table 7) a subject was called an Excellent Focuser in a particular 
marginal cell if at least six out of his nine protocols had been labeled 
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as perfect focusing. A Good or Poor Focuser had at least six "focusing" 
(either perfect or with error) protocols out of the nine. Good Focusers 
had a majority of perfect protocols, while Poor Focusers had mostly 
focus ing-with-error protocols. A similar six-out-of-nine collapsing rule 
was used to assign subjects to the Subsetter subgroup. Subjects in the 
Inconsistent subgroup did not have a classifiable set of protocols (i.e., 
did not have either six "focusing" or six "subsetting" protocols). The 
"Overall" strategy subgroup assignments were based on 27 protocols; again, 
a two-thirds (18 out of 27) rule was used. The subgroup assignments for 
the troubleshooting task (Table 8) were made following the same two-thirds 
rule. In this case, marginal subgroup assignments were based on six protocols, 
the overall assignment based on 18 protocols. 

The frequency distributions presented in both Tables, of course, would 
have been affected by the adoption of different collapsing rules. For 
example, if a nine-out-of-nine criterion had been adopted for the concept- 
identification task, the Inconsistent subgroup would have been larger. 
Also, the Inconsistent groups are conceptually somewhat different in the 
two tasks. For concept identification, an inconsistent subject had protocols 
distributed between focusing and subsetting. However, in the troubleshooting 
task, a subject could be called inconsistent if his protocols were evenly 
split between "good focusing" and "poor focusing" as well as between focusing 
and subsetting. Despite these limitations on interpretation, several 
interesting points can be abstracted from these Tables. 

For subjects in the concept-identification task (Table 7), the manipu- 
lation of perceptual complexity did not affect the frequency of occurrence 
of different strategies. However, the number-of-dimensions manipulation 
substantially reduced the number of Excellent Focusers, while the number of 
Subsetters increased. Although not discernable from the data presented in 
Table 7, strategy "shifts" by individual subjects occurred for the most 
part in the expected direction: some Excellent Focusers moved to Good or 
Poor Focusers, and some Good and Poor Focusers moved to subsetting. Sub- 
setters were very consistent; that is, very few subjects shifted from 
subsetting to a different strategy. 
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Similar trends can be observed for the troubleshooting data shown in 
Table 8* In this case, both task-characteristic manipulations reduced the 
number of Excellent and Good Focusers. (Frequencies for these two groups 
were combined due to the low incidence of subjects meeting the Excellent- 
Focuser criteria for the more difficult problems.) The number of Poor 
Focusers increased dramatically across the formal difficulty manipulation* 
Again, Subsetters were very consistent across both task-characteristic 
manipulations. 

Attention was next directed toward whether strategies had any impact 
on performance. In the concept-identification task, this question was 
addressed directly by comparing the proportion of problems solved using a 
focusing strategy to the proportion solved using a subsetting strategy. 
These proportions are presented in Table 9 as a function of task-characteristic 
marginals. As in apparent from these data, a focusing strategy was far 
more effective than a subsetting strategy for the solution of the problems 
at all levels of task characteristics. This result conforms to both 
theoretical expectations and previous empirical findings (e.g., Wandersman & 
Wandersman, 1973). Likewise (although not presented here), focusing was 
superior to subsetting on the other performance measures obtained in the 
study. 

A more detailed examination of the impact of strategies on performance 
was undertaken for the troubleshooting task. In this case, it was not 
obvious what strategy, was "better" for the dependent variables obtained. 
Similarly, this detailed examination^V^'as important to "validate" the 
procedure for generalizing concept-identification strategies to a different 
task. The method chosen for this analysis was to compare performance of 
subgroups of subjects using different strategies on two dependent variables: 
mean log time to solution and arcsin, square root efficiency per trial. 
Mean performance for each strategy subgroup on these variables as a function 
of task-chfiracteristic marginals is presented in Table 10. Tests of the 
statistical significance of between-subgroup contrasts are presented in 
Table 11. 
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These analyses produced some surprising results. For example, it was 
not anticipated that Poor Focusers would be the slowest and least efficient 
subgroup; one might have expected a speed-accuracy trade-off to have 
operated (i.e., the subjects who were most accurate, the Excellent and 
Good Focusers, could have sacrificed speed in order to locate logically 
efficient tests). Despite this and other unanticipated findings, the 
general picture provided by these analyses was that the level of performance 
achieved on a given problem by subgroups of subjects was clearly related 
to the kind of strategy employed. Furthermore, this influence was moderated 
by the dependent variable examined and by the task-characteristic manipu- 
lations. For example, Inconsistents did not differ from Subsetters with 
respect to the time-to-solution or efficiency-per-trial measures for the 
easiest problems. However, the Subsetters were significantly less efficient 
than Inconsistents on the more difficult problems (see Table 11). 

Summary, . Although interesting in themselves and worth pursuing in 
their own right, these particular strategy-performance analyses will not 
be discussed further in the present report. The purpose of these analyses 
was to demonstrate that a strategy-performance relationship existed, the 
precise nature of the relationship being a subsidiary issue. Likewise, 
the nature of the relationship could have been modified by different 
subgroup-assignment rules. As will be described below, subgroups were 
reconstituted during the course of further analyses. Therefore, the main 
conclusions to be drawn from these strategy-performance analyses are, first, 
that the labeling scheme could be effectively used to assign strategy codes 
to individual protocols, and second, that choice of strategy certainly 
impacts on performance. Accordingly, the next issue addressed was the 
relationship between strategy and ability. Are subjects who employ a 
particular strategy discriminable on the basis of their ability profiles 
from subjects who adopt alternative problem-solving approaches? 

Strategies and Abilities '"-^ 

To address the strategy-ability relationship a stepwise discriminant 
analysis was performed. Factor scores for each subject in the various 
strategy subgroups (defined independently for each task characteristic level) 
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served as predictors in the discriminant analyses. Again, despite the 
level of detail of the analysis, the basic question was straightforward: 
can subgroups of subjects categorized on the basis of strategy usage be 
discriminated on the basis of their factor scores? Tables 12 and 13 
present the results of the discriminant analyses for the two experiments. 
These tables provide, for the overall strategy classification and for each 
task characteristic subgroup, the following information: the abilities 
which significantly differentiated subgroups, Wilks* lambda (the multivariate 
analogue to an £-ratio), the proportion of classification variance accounted 
for, and the between-groups discrimination matrix. 

For the most part, subgroups were discriminable on all versions of 
the two tasks. Discrimination was greater in the concept-identification 
study than in the troubleshooting case, as indicated by the greater pro- 
portion of variance accounted for in the former. 

An example will serve to demonstrate the type of information obtained 
and the inferences that can be drawn. Consider the discriminant 
analysis predicting the overall classification in the concept-identification 
task. Table 12 shows that when the Syllogistic Reasoning, Associative 
Memory, Flexibility of Closure/Spatial Scanning, and Speed of Closure 
factors are included in the discriminant function. Excellent Focusers are 
discriminable from Subsetters. In fact, the Excellent Focusers have sub- 
stantially higher mean factor scores on all four factors; however, this 
directional consistency was not always the case. That is, the Subsetter 
and Inconsistent subgroups are also discriminable on the basis of these four 
factors; however, Subsetters have higher mean factor scores on Syllogistic 
Reasoning and Associative Memory, lower scores on Speed of Closure, and 
practically identical scores on Flexibility of Closure/Spatial Scanning. 

Summary . The primary purpose of these analyses was to demonstrate 
that the subgroups were discriminable. As was the case in the strategy- 
performance analyses, these discriminant analyses are interesting in their 
own right, but further discussion is not warranted at this time. Again, 
the specific relationships uncovered may vary as a function of the rules 
used to generate the subgroups. While the present subgroups were interesting 
from a theoretical and methodological perspective, other possible subgroupings 
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are more important for the purposes of subsequent analyses. Several 
sortings and reanalyses of the resulting subgroups were conducted during 
the course of the project. One of these was used to investigate, in detail, 
the task characteristic-ability-strategy relationship. Results from this 
central analysis are presented and discussed in the following section. 

Abilities, Strategies, and Task Characteristics 

To recapitulate briefly, the analytical steps employed in order to 
establish the role of strategy as a mediator between abilities and task 
characteristics were as follows. First, a priori criteria were established 
to classify subjects into strategy subgroups. Second, it was determined 
that the constitution of the subgroups impacted on performance. Third, 
it was determined that this strategy subgroup sorting was related to 
abilities. In light of these prior outcomes, the final step is to determine 
the correlations between the ability-factor scores and performance as a 
function of variations in task characteristics for each strategy subgroup. 
This analytical procedure was performed for both the concept-identification 
and the troubleshooting tasks. 

Concept-Identification task . On the basis of the analyses presented 
above, six strategy subgroups of subjects were established. These six 
subgroups consisted of Good Focusers, Fair Focusers, Poor Focusers, Partial 
Subsetters, Subsetters, and Inconsistent. The criteria for subject assign- 
ment were similar to those previously discussed; the primary difference 
was the creation of a subgroup of subjects who attempted a subset strategy 
at some point in the experiment but did not maintain it over a sufficient 
number of problems to be categorized as Subsetters. 

Tables 14 and 15 present, respectively, the mean proportion of problems 
solved by each subgroup as a function of task characteristics, and the 
performance discrimination matrices. It is clear that the subgroups could 
be discriminated in terms of performance. 

Tables 16 and 17 present, respectively, the mean factor scores and the 
factor-score discrimination matrices for the strategy subgroups. Although 
there are few obvious subgroup mean differences, some general trends can 
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Table 15 

Performance Discrimination Matrices: Differences in Proportion of 
Problems Solved Among Strategy Subgroups as a Function of 
Task Characteristics in Concept Identification ^ 



Strategy Subgroup 



Task 
Characteristic 



Comparison 
Subgroup 



FG 



FF 



FP SS 



PS 



No. of Dimensions 



FF 


.46 








FP 


3.95** 


3.36** 






SS 


3.65** 


3.46** 


1.79 




PS 


2.35** 


2.08* 


.22 


1.77 


I 


3.41** 


2.96** 


.02 


1.76 


FF 


1.79 








FP 


5.79** 


1.26 






SS 


5.48** 


2.82** 


2.36* 




PS 


5.40** 


1.90 


1.11 


1.38 


I 


5.17** 


2.18* 


1.53 


.87 


FF 


2.72* 








FP 


5.37** 


1.38 






SS 


8.32** 


4.83** 


3.98** 




PS 


5.93** 


3.23** 


2.32* 


1.19 


I 


5.46** 


2.57* 


1.59 


2.05* 



^ Subgroup Abbreviations are: 

FG: Good Focuser SS: Subsetter 

FF: Fair Focuser PS: Part Subsetter 

FP: Poor Focuser I: Inconsistent 

Table entries are t_-values. 

* P:^.05 

** PS. 01 
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Table 15 (Continued) 

Performance Discrimination Matrices: Differences in Proportion of 
Problems Solved Among Strategy Subgroups as a Function of 
Task Characteristics in Concept Identification ^ 

Strategy Subgroup 



Subgroup Abbreviations are: 



Task 
Characteristic 


Comparison 
Subgroup 


rb 


rr 


CD 

rr 


SS 


PS 


Level of Pprcpntual 
Complexity 














A 


FF 
FP 


1 .93 
5.72** 


3.98** 












7 AAifk 


D. OU^^ 


3 27** 








r o 


0. J7 1 


A OCieie 
4 .00 


CA 
.04 


^.d5* 






T 


Dm Hu 


A OMcie 


QC 


I /* 


.00 


B 


FF 
FP 


1.59 
6.43** 


1.97 










SS 


4.03** 


2.54* 


1.49 








PS 


4.31** 


2.43* 


1.21 


.39 






I 


4.89** 


2.49* 


1.14 


.59 


.20 


C 


FF 
FP 


2.50* 
4.09** 


.24 










SS 


7.26** 


3.71** 


4.15** 








PS 


4.33** 


1.68 


1.72 


1.82 






I 


4.78** 


1.46 


1.51 


2.47** 


.41 



FG 
FF 
FP 



Good Focuser 
Fair Focuser 
Poor Focuser 



SS 
PS 
I 



Subsetter 
Part Subsetter 
Inconsistent 



Table entries are _t-values, 

* p ^ .05 
** piz .01 
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Table 17 

Ability Discrimination Matrices: Differences in Factor Scores 
Among Strategy Subgroups in Concept Identification ^ 



Strategy Subgroup 



Factor 


Comparison 
Subgroup 


F6 


FF 


FP 


33 PS 


Flexibility of 
Closure/Spatial 
Scanning 


FF 
FP 


.76 
2.61* 


1.01 








SS 


3.45** 


1.68 


.99 






PS 


1.31 


.42 


.52 


1.22 




I 


3.02** 


1.37 


.55 


.42 .90 


Associative Memory FF 


.58 










FP 


.22 


.47 








SS 


2.94** 


1.29 


2.90** 






PS 


1.15 


.41 


1.05 


.84 




I 


.30 


.37 


.12 


2.40** .90 


Perceptual/ 
Cognitive 
Speed 


FF 
FP 


.73 - 
.30 


.99 








SS 


1.67 


2.07* 


1.52 






PS 


.52 


.31 


.86 


2.17* 




I 


1.06 


.21 


1.39 


2.54* .55 



^ Subgroup abbreviations are: 

F6: Good Focuser SS: Subsetter 

FF: Fair Focuser PS: Part Subsetter 

FP: Poor Focuser I: Inconsistent 

Table entries are _t-values* 

* p ^.05 
** p^: -01 
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Table 17 (Continued) 

Ability Discrimination Matrices: Differences in Factor Scores 
Among Strategy Subgroups in Concept Identification ^ 



Strategy Subgroup 



Factor 


Comparison 
Subgroup 


F6 


FF 


FP 


SS 


PS 


Svl 1 oai <;tic 
Reasoning 


FF 
FP 


.70 
.82 


1.17 










SS 


3.19** 


2.75* 


2.91** 








PS 


1.49 


1.61 


.97 


1.77 






I 


1 "id 


1 . Oo 


1 • uo 


1 •oC 


on 

. CO 


Speed of 
Closure 


FF - ■ 
FP 


.53 
.28 


.73 






• 




SS 


.07 


.47 


.10 








PS 


2.07* 


2.11* 


1.91 


1.52 






I 


.82 


1.10 


.60 


.50 


1.28 


Induction 


FF 
FP 


.26 

1.39 


.97 










SS 


1.19 


.81 


.15 








PS 


1.22 


.92 


.21 


.30 






I 


1.57 


1.21 


.46 


.55 


.16 



^ Subgroup abbreviations are: 



F6 
FF 
FP 



Good Focuser 
Fair Focuser 
Poor Focuser 



SS 
PS 
I 



Subsetter 
Part Subsetter 
Inconsistent 



* 
** 



Table entries are t^-values. 

p 4 .05 
P^ .01 
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be discerned. Good and Fair Focusers are superior to all other subgroups 
on most factors. Subsetters are substantially below the other subgroups 
on several factors, as is the Inconsistent subgroup. 

Correlations between factor scores and performance for each of the 
strategy subgroups as a function of task characteristics are presented in 
Table 18. These correlations are analogous to those shown in Table 6 above; 
however, since that analysis was performed on the entire pool of subjects 
rather than on smaller (and presumably more homogeneous) subgroups, the 
magnitude of the correlations is not comparable. 

There are several trends indicated by these data that may help in 
explaining why ability-task characteristic relationships previously reported 
were difficult to interpret. At the most general level, it is clear that 
different abilities correlated with performance for the various subgroups. 
For example. Flexibility of Closure/Spatial Scanning (FC/SS) correlated 
highly with performance for the Subsetters, but not for other subgroups. 
Likewise, Induction was important for Fair Focusers but not for anyone 
else. Hence, if all subgroups were combined, one would expect ambiguous 
results. Similarly, even for abilities predicting performance for different 
subgroups, the pattern of involvement across task manipulations differs for 
the different strategies. For example. Associative Memory (ASCMEM) has 
high correlations with performance for Fair Focusers, Part Subsetters, and 
Subsetters across the "number-of-dimensions" task manipulation. However, 
the correlations increase consistently for the Fair Focusers; the ability 
"drops out" for the Subsetters at the most difficult problem level; while 
for the Part Subsetters it increases dramatically at the most difficult 
level. Again, pooling these three subgroups would result in a possibly 
misleading pattern of involvement which would not reflect any of the subgroup 
patterns. To further illustrate this last point, consider the Inconsistent 
subgroup. These subjects (by definition) did not use a strategy consistently 
from one problem to the next. As might be expected, there are no consistent 
patterns of correlations across either task manipulation. 

Another outcome from these correlations that would limit the inter- 
pretability of correlations obtained from pooled subjects is that abilities 
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Table 18 

Correlations Between Ability- Factor Scores and Concept 
Identification Performance as a'f^unction of Variations 
in Task Characteristics for Each Strategy Subgroup ^ 

Tdisk Characteristic 















Level of 




Strategy 


Ability- 


No. 


of Dimensions 


Perceptual Complexity 




r aVrf uUi 


4 


6 


8 


A 


B 


C 


Good Focusers 


FC/SS 


-08 


25 


-20 


-05 


14 


09 




ASCMEM 


23 


-58 


17 


10 


-29 


-41 




PERCSP 


-34 


16 


. -12 


-20 


-19 


23 




SYLREA 


-37 


07 


03 


-08 


02 


-04 




SPCLOS 


-n 


-10 


14 


-25 


-04 


07 




IND 


05 


25 


OS 


24 


07 


12 


Fair Focusers 


FC/SS 


-35 


no 

-08 


-23 


-20 


-28 


-08 




ASCMEM 


11 


33 


49 


29 


68 


16 




PERCSP 


39 


57 


17 


14 


44 


52 




SYLREA 


15 


-20 


-51 


01 


-44 


-30 




SPCLOS 


-11 


47 


07 


42 


-03 


48 




IND 


37 


67 


21 


42 


30 


71 


Poor Focusers 


FC/SS 


22 


32 


-11 


-16 


24 


34 




ASCMEM 


03 


22 


-03 


00 


08 


12 




PERCSP 


36 


02 


15 


02 


33 


24 




SYLREA 


-08 


11 


02 


-14 


05 


24 




SPCLOS 


13 


-14 


04 


-01 


09 


-01 




IND 


-01 


-20 


31 


08 


-02 


13 



s Correlations rounded to two places, decimals omitted. 

^ Abbreviations for Ability-Factors: 

FC/SS: Flexibility of Closure/Spatial Scanning IND: Induction 
ASCMEM: Associative Memory SYLREA: Syllogistic Reasoning 

PERCSP: Perceptual/Cognitive Speed SPCLOS: Speed of Closure 
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Table 18 (Continued) 

Correlations Between Ability-Factor Scores and Concept 
Identification Performance as a Function of Variations 
in Task Characteristics for Each Strategy Subgroup ^ 



Task Characteristic 















Level of 




Strategy 


Ability- 


No. 


of Dimensions 


Perceptual Complexity 


Subgroup ^ 


Factor 


4 


6 


8 


A 


B 


c 


Subsetters 


FC/SS 


53 


64 


74 


53 


79 


64 




ASCMEM 


44 


65 


-10 


32 


31 


48 








-24 


-42 


-25 


-49 


-44 




SYLREA 


-01 


-24 


07 


13 


-26 


00 




SPCLOS 


32 


26 


07 


44 


10 


20 




IND 


-13 


-11 


-02 


-09 


-15 


-04 


Part Sub- 
















setters 


FC/SS 


-20 


10 


-23 


-59 


07 


-01 




ASCMEM 


49 


47 


74 


53 


55 


66 




PERCSP 


-37 


07 


-02 


-17 


03 


-18 




SYLREA 


47 


10 


16 


26 


15 


29 




SPCLOS 


54 


08 


53 


50 


49 


24 




IND 


17 


25 


10 


-03 


36 


10 


Inconsistent 


FC/SS 


10 


-14 


-10 


16 


-16 


-22 




ASCMEM 


05 


45 


20 


36 


28 


16 




PERCSP 


-27 


25 


28 


-14 


19 


40 




SYLREA 


01 


• 04 


21 


00 


24 


06 




SPCLOS 


-25 


-49 


-37 


-31 


-46 


-43 




IND 


-16 


19 


15 


02 


27 


00 



^ Correlations rounded to two places, decimals omitted. 

^ Abbreviations for Ability-Factors: 

FC/SS: Flexibility of Closure/Spatial Scanning IND: Induction 
ASCMEM: Associative Memory SYLREA: Syllogistic Reasoning 

PERCSP: Perceptual/Cognitive Speed SPCLOS: Speed of Closure 
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apparently operate differentially for different subgroups within a given 
task characteristic level. For example, consider the Perceptual Speed 
(PERCSP) factor for the 4-diinensional problems. Correlations are substantial 
for all subgroups; however, these correlations are positive for the Fair 
Focusers and Poor Focusers, but negative for the other groups. The net 
result upon combining subgroups (Table 6) was that this factor had an 
essentially zero correlation with performance. A final outcome which would 
limit the interpretability of pooled results is that even for subgroups 
using the same general strategy, different abilities were related to 
performance at different levels of the task-characteristic manipulations. 
For example, consider the Good, Fair, and Poor Focusers and their corre- 
lational patterns at the "C" level of perceptual complexity, clearly, the 
patterns of correlations are different, although the same general strategy 
was used. Thus, there is no single expected pattern of correlations for 
the general strategy of focusing; if one knew that a given subject was a 
focuser (or even that he was a Good Focuser) on the 4-dimension problems, 
one could not predict his performance on a variant of the task. It might 
be possible to predict from his ability profile that he would likely continue 
to be a Good Focuser (or change to a Fair or Poor Focuser) and, from the 
predicted strategy, predict performance on the modified task. Similarly, 
one could not predict performance simply from a knowledge of a subject's 
ability without knowing which strategy he used in other versions of the 
task. 

Troubleshooting. The analytic procedures described above for the 
concept-identification study were carried out for the troubleshooting task. 
There were three principal reasons why this was done. First, it was an 
attempt to validate the methodological and analytic procedures developed 
to ascertain the mediational role of strategy for ability-performance 
relationships. The second reason was to gain support for the general con- 
clusions drawn from the concept-identification analyses. Finally, there 
were several specific issues related to strategy types, dependent variables, 
and task characteristics which could be addressed directly through these 
analyses. 
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The first step was again to generate subject subgroups on the basis 
of their protocol strategies. The subgroups selected for these analyses 
were as follows: 

1. Focusers (F): subjects who had mostly excellent or good focusing 
protocols for each level of task manipulation. 

2. Good-to-Poor Focusers (GPF): subjects who were classified as 
Excellent/Good Focusers on the easiest problems, but became Poor 
Focusers on the more difficult problems. 

3. Early-to-Inconsistent Focusers (EIF): subjects who were 
classified as Excellent/Good Focusers on the easiest problems, 
but could not be classified as Good or Poor Focusers thereafter. 

4. Change Focusers (CF): a pooling of the GPF and EIF subgroups. 

5. Subsetters: subjects who had mostly subset protocols for all 
levels of task manipulations. 

6. Inconsistent (I): subjects who were unclassifiable by the preceding 
criteria (typically, I subjects had several subset protocols 
randomly distributed across task dimensions). 

The next steps were to assess the impact of this categorization upon 
performance and ability profiles. Two performance measures were selected 
for analysis: mean log time to solution, and arcsin, square-root efficiency 
per trial. These measures were chosen because they were basically inde- 
pendent of the protocol classification rules. Mean levels of performance 
for each subgroup as a function of task characteristics are presented in 
Table 19. Subgroup pairwise contrasts of these means were performed for 
each cell. (A table of these comparisons will not be presented due to 
the large number of jt-values that would have to be presented.) As was 
true for the concept-identification task, most strategy subgroups differed 
from one another on performance, with the exceptions of the "A" level of 
perceptual complexity on the time-to-solution measure, and scattered subgroup 
combinations within each cell. The mean ability scores for each subgroup 
are presented in Table 20. Subgroup pairwise contrasts of these means were 
performed. Again, subgroups were found to differ on several factors. Thus, 
the two necessary conditions to establish "strategy" as a mediator have been 
met; strategies are related to performance and abilities. 
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Table 19 

Mean Performance as a Function of Task 
Characteristic and Strategy in Troubleshooting 



Task Characteristic 



Level of Difficulty Level of 

Strategy Classification Perceptual Complexity 

(Group size) 1 2 3 A B C 



Log Time to Solution 



Focusers (21) 


4 


89 


5.03 


5 


.67 


4 


.98 


5.20 


5.41 


Good- to- Poor 
















Focusers (14) 


4 


99 


5.46 


6 


.02 


5 


23 


5.52 


5.71 


Early-to-Inconsistent 
















Focusers (27) 


4. 


85 


5.14 


5 


.74 


5 


00 


5.22 


5.52 


Change Focusers (41 ) 


4. 


90 


5.25 


5 


84 


5 


08 


5.33 


5.58 


Subset (20) 


5. 


09 


5.21 


5 


77 


5 


13 


5.29 


5.65 


Inconsistent (40) 


5. 


05 


5.14 


5 


80 


5. 


04 


5.31 


5.64 



Arcsin, Square Root Efficiency Per Trial 



Focuser 


1 


.10 


1.15 


.98 


1 .09 


1.09 


1.05 


Good-to-Poor 
















Focuser 


1 


.04 


.82 


.74 


.89 


.87 


.84 


Early-to-Inconsistent 
















Focuser 


1 


.11 


1 .02 


.90 


1.04 


1 .02 


.97 


Change Focuser 


1 


.08 


.95 


.85 


.99 


.97 


.93 


Subset 




.90 


.86 


.76 


.87 


.85 


.79 


Inconsistent 




.92 


.92 


.76 


.90 


.89 


.81 
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Table 20 

Mean Ability-Factor Scores for Strategy 
Subgroups in the Troubleshooting Experiment 



Ability Factors ^ 



strategy Subgroup 


FC/SS 


SYLREA 


ASCMEM 


PERCSP 


IND 


Focusers 


.432 


.304 


-.094 


-.425 


.436 


Good- to- Poor 
Focusers 


-.242 


-.486 


.505 


-.434 


-.273 


Earl y-to- Inconsis- 
tent Focusers 


.330 


.410 


.253 


.315 


-.173 


Change Focusers 


.135 


.104 


.339 


.059 


-.207 


Subsetters 


-.132 


-.203 


-.322 


.237 


.112 


Inconsistent 


-.445 


-.112 


-.124 


-.020 


-.140 



^ Abbreviations for Ability Factors: 

FC/SS: Flexibility of Closure/Spatial Scanning 

SYLREA: Syllogistic Reasoning 

ASCMEM: Associative Memory 

PERCSP: Perceptual Speed 

IND: Induction 
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Table 21 presents the correlations between factor scores and performance 
for each subgroup as a function of variations in task characteristics. For 
the most part, the general conclusions reached from the parallal concept- 
identification analysis are supported in the troubleshooting task. First, 
different abilities correlated with performance for the various subgroups, 
although there was more consistency in this study than in the concept- 
identification study. For example. Induction was correlated with the time 
measure for all subgroups except the Focusers and Inconsistents. Also, a 
given ability had different patterns of involvement for different subgroups, 
which when combined, could lead to ambiguous or misleading conclusions. 
For example, the Associative Memory Factor had substantial negative corre- 
lations with the time measure for the GPF and CF groups, but high positive 
correlations for the Subsetters. Similarly, FC/SS correlations with the 
time measure increase across the difficulty manipulation for the EIF and CF 
subgroups, stays high and positive for the F, SS, and I subgroups, and is 
basically uncorrected with the GPF subgroup. These differential outcomes 
are totally obscured when correlations are examined across all subjects 
(Table 4). 

An additional interesting result is that the abilities correlate 
differently with the time and efficiency measures for each subgroup. For 
example. Induction, while not correlated with the time measure for the I 
subgroup, is highly related to their efficiency scores; the opposite is 
true for the Subsetters (i.e., correlated with time but not with efficiency). 
Perhaps more impressively. Syllogistic Reasoning is in general negatively 
related to the time measure for Subsetters, but has high positive corre- 
lations with efficiency. If Table 21 is compared to Table 4, one can see 
that the overall correlations are a function of the combination of the 
subgroup correlations for each cell. Therefore, the interpretation of 
the overall results must be modified by consideration of subgroup membership 
and hence of strategy employed. 
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IV. CONCLUSIONS 



The experimental-correlational studies described in the present 
report, together with those reported upon previously by Zimmerman (1954) 
and Fleishman (1957), were conducted to explore ability-task interactions 
in situations representing a spectrum of human performance. The earlier 
studies, involving visual ization-of-maneuver and response-orientation 
tasks, investigated the visual -perceptual and perceptual -motor domains. 
The later studies, concerned with auditory target identification, elec- 
tronic troubleshooting, and problem solving, have delved into auditory- 
perceptual and perceptual-cognitive tasks of current interest to the Navy. 

In each of these investigations the general approach was to vary 
tasks along one or two specified physical dimensions and to administer 
these task variations to groups of subjects who also received a battery 
of reference tests known to sample certain more general abilities (e.g., 
syllogistic reasoning, associative memory, perceptual speed, etc.). Corre- 
lations between these ability constructs and measures of performance on 
variations of the criterion task serve to specify the ability requirements, 
and changes in these requirements, as a function of criterion task varia- 
tions. 

Considered jointly, these experimental -correlational studies have 
provided convincing support for the validity of the ability constructs 
sampled during the course of the various studies. Ability scores do 
correlate with criterion task performance and do account for sizeable 
portions of the variance in those measures— variance which is typically 
ascribed to individual differences among operators. Furthermore, those 
abilities accounting for individual differences in performance on one task 
are not necessarily the same as those involved in other, obviously dis- 
similar tasks. This recurrent finding supports the contention that abilities 
are sensitive to differences among tasks, making them more precise descrip- 
tors of performance than the broader rubrics (e.g., mental, motor; cognitive, 
noncognitive; etc.) in vogue not so long ago. 
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In many respects, the current program of research assumed these out- 
comes from the start. If personnel requirements are to be stated in terms 
of abilities, such constructs must in fact contribute to the prediction 
of operator performance. Similarly, to the extent that diverse tasks 
(as defined in terms of ability requirements) are considered, differential 
ability- task relationships must be demonstrable. Given these assumptions, 
therefore, the basic thrust of the research program was to test their 
limits. Interest lay in exploring how the ability requirements associated 
with a given task might change as physical "stimulus" features of the 
task were varied in small and systematic increments. One of four outcomes 
was theoretically possible: 

1. no change in ability requirements, either in terms 

of those involved in the original version of the task 
(i.e., those exceeding a specific correlational value), 
or in terms of their degree of involvement (i.e., 
an increase or decrease in the correlation coefficient); 

2. a systematic change (increase or decrease) in the 
involvement of the relevant ability(ies); 

3. a systematic increase in the involvement of one 
or more abilities and a simultaneous decrease in 
the importance of others; or 

4. changes in involvement which did not appear to be 
systematic over incremental task variations. 

The Zimmerman (1954) and Fleishman (1957) studies demonstrated the third 
kind of outcome, the Wheaton et al . (1973) passive-sonar study yielded 
the second kind of outcome, while the Rose et al. (1974) troubleshooting 
and Fingerman et al . (1975) concept-identification studies yielded the 
fourth type of outcome. 

Failure to produce the first outcome was, of course, primarily due 
to the experimenters' ingenuity in selecting task characteristics and incre- 
mental values which would impact on task performance and upon ability 
requirements. For purposes of the research such variations were desirable 
since they served to demonstrate that an effective incremental change 
had in fact occurred. 

68 



7S 



The second outcome, as demonstrated in the Wheaton et aK study, has 
implications fojr ability assessment techniques (Theologus et al., 1970) as 
well as for personnel decisions based on such assessments. It suggests 
that there are certain kinds of task variations which simply influence what 
might be termed the conditions of performance rather than the nature of 
the performance itself. The abilities contributing to individual differences 
in performance on the original task continue to do so, but become more 
important as the conditions under which performance occurs become more 
demanding. Such a possibility must be kept in mind by the abilities 
analyst who attempts to decermine the ability requirements for the task 
variation based upon his knowledge of the requirements for the original 
version. Similarly, the personnel decision maker, given certain other 
constraints, should be prepared to retain his selection instruments 
but to raise his cutoff criterion. 

The third outcome, as demonstrated in the Fleishman and Zimmerman 
studies has important and different implications. These studies demonstrate 
that subtle variations can be induced which actually change the nature of 
the task, such transformations being defined in terms of new ability 
requirements. Again, ability analysts must be sensitive to such vari- 
ations if they are to avoid erroneous extrapolations from the ability 
requirements associated with base-line task conditions. Individuals 
responsible for selection will have to modify their instruments. 

Given these outcomes the issue becomes one of differentiating between 
two general kinds of task variation. One, which seems extrinsic to the 
task itself, serves to change conditions under which the task is performed 
without altering the set of abilities contributing to individual differences 
in performance. The other, which appears to be an integral part of the 
task, serves to modify those abilities accounting for individual differences. 
The Rose et al. and Fingerman et al . studies included task-characteristic 
manipulations believed to represent both extrinsic and intrinsic task varia- 
tions. Levels of difficulty, as defined in those studies, entailed an 
intrinsic variation while perceptual complexity represented a background 
or extrinsic variation. 

69 



79 



Unfortunately, both of these studies produced the fourth outcome. 
Changes in ability requirements were manifest, but these changes were not 
of the systematic variety anticipated (second and third outcomes). Thus, 
it became apparent that intrinsic and extrinsic variations could not 
be defined in terms of morphological features of a task. Rather, the 
distinction between an intrinsic and extrinsic variation must be a 
function of the subject* s perception of the change in a physical task 
characteristic. If the subject perceives such a change as representing 
a change in the way he must now perform the task, that change can be 
said to represent an intrinsic task variation.^ . Conversely., if the 
task manipulation is not so perceived, it represents an extrinsic varia- 
tion. This reasoning was explored on a post hoc basis by examining 
the relationship between task variations and the problem-solving strategies 
employed by individual subjects. 

Reanalysis of the Rose et al . and Fingerman et al . studies in terms 
of these concepts served to clarify the circumstances which produced 
the fourth type of outcome. In light of these findings, the implications 
for ability assessment and for personnel decision making are significant. 
In situations potentially representing intrinsic variations in a task, 
estimates of the ability profiles associated with different problem- 
solving approaches or strategies will be required. Furthermore, methods 
will be needed in order to identify individuals who will adopt one of 
several strategies as well as those who will change their strategy in 
response to a change in the task. The goal of future research will be to 
supply such methods. 



70 

80 



REFERENCES 



Bruner, J. S., Goodnow, J. J., & Austin, 6. A. A study of thinking . 
New York: Wiley, 1956. 

Buros, 0. K. , (Ed.) The sixth mental measurements yearbook . Highland 
Park, New Jersey: Gryphon Press, 1965. 

Farina, A. J., Jr., & Wheaton, 6. R. Development of a taxonomy of human 
performance: The task characteristics approach to performance pre- 
diction . Report No. AIR-2035-2/71-TR7, February, 1971. American 
Institutes for Research, Washington, D. C. 

Fingerman, P. W. , Eisner, E., Rose, A. M. , Wheaton, 6. R., & Cohen, F. 
Methods for predicting job-abiVity requirements: III. Ability 
requirements as a function of changes in the characteristics of a 
concept identification task . Report No. AIR-31300-4/75-TR, April, 
1975, American Institutes for Research, Washington, D. C. 

Fleishman, E. A. Factor structure in relation to task difficulty in psy- 
chomotor performance. Educational and Psychological Measurement , 
1957, U, 522-532. 

Fleishman, E. A., & Bartlett, C. J. Human abilities. Annual Review of 
Psychology , 1969, 20, 349-380. 

Fleishman, E. A., & Friedman, M. P. The development of the Air Force 

Aural Code Test . Lackland Air Force Base, Texas: Air Force Personnel 
and Training Research Center, November 1957. (Research Report AFPTRC- 
TN-57-131, AD-146 404). (a) 

Fleishman, E. A., & Friedman, M. P. Some standardized tests of auditory- 
perceptual abilities . Lackland Air Force Base, Texas: Air Force 
Personnel and Training Research Center, November 1957. (Research 
Report AFPTRC-TN-57-132, AD-146 405). (b) 

Fleishman, E. A., Kinkade, R. G. , & Chambers, A. N. Development of a 

taxonomy of human performance: A review of the first year's progress . 

Report No. AIR-726-11/68-TPR1 , November 1968. American Institutes for 
Research, Washington, D. C. 

Fleishman, E. A., & Spratte, J. G. The prediction of radio operator success 
by means of aural tests . Lackland Air Force Base, Texas: Air Force 
Personnel and Training Research Center, November 1954. (Technical 
Report AFPTRC-TR-54-66). 

Fleishman, E. A., & Stephenson, R. W. Development of a taxonomy of human 
performance: A review of the third year's progress^ Report No. 
AIR-2035-9/70-TPR3, September 1970. American institutes for Research, 
Washington, D. C. 

71 



81 



Fleishman, E. A., Teichner, W. H., & Stephenson, R, Development of 
a taxonomy of human performance: A review of the second year's pro- 
gress . Report No. AIR-726-1/70-TPR2, January 1970. American Institutes 
tor Research, Washington, D. C. 

French, J. W. , Ekstrom, R. B., & Price, L. A. Manual for kit of reference 
tests for cognitive factors . (Rev. ed.) Princeton, New Jersey: 
Educational Testing Service, 1963. 

Gregg, L. W., & Simon, H. A. Process models and stochastic theories of 
simple concept formation. Journal of Mathematical Psychology , 
1967, 4, 246-276. 

Levine, J. M., Greenbaum (Kramer), 6., & Notkin, E. R. The effect of 
alcohol on human performance: A classification and integration of 
research findings . Report No. R73-4, May 1973, American Institutes 
for Research, Washington, D. C. 

Levine, J. M. , Romashko, T., & Fleishman, E. A. Development of a taxonomy 
of human performance: Evaluation of an abilities classification 
system for integrating aixl generalizing research findings . Report 
No. AIR-726-2035-9/71-TR12, September 1971, American Institutes for 
Research, Washington, D. C. 

Levine, M. Hypothesis behavior by humans during discrimination learning. 
Journal of Experimental Psychology , 1966, 71, 331-338. 

Levine, M. Human discrimination learning: The subset-sampling assumption. 
Psychological Bulletin , 1970, 74, 397-404. 

Levine, M. A cognitive theory of learning: Research on hypothesis testing . 
Hillsdale, New Jersey: Lawrence Erlbaum Associates, 1975. 

Postman, L. A pragmatic view of organization theory. In E. Tulving, & 
W. Donaldson (Eds.), Organization of memory . New York: Academic 
Press, 1972. 

Rose, A., Fingerman, P. W. , Wheaton, G. R., Eisner, E. J.s & Kramer, 6. 

Methods for predicting job-abili_ty requirements: II. Ability require- 
ments as a function of changes in the characteristics of an electronic 
fault-finding task . Report No. AIR-31300-8/74-TR, August 1974, 
American Institutes for Research, Washington, D. C. 

Theologus, 6. C, & Fleishman, E. A. Development of a taxonomy of human 
performance: Validation study of ability scales for classifying 
human tasks . Report No. AIR-726/2035-4/71-TR10, April 1971. 
American Institutes for Research, Washington, D. C. 

Theologus, 6. C, Romashko, T. , & Fleishman, E. A. D evelopment of a taxo- 
nomy of human performance: Feasibility study of ability dimensions 
for classifying tasks . Report No. AIR-726-1/70-TR5, January 1970. 
American Institutes for Research, Washington, D. C. 



72 



Tulving, E., & Donaldson, W. (Eds.) Organization of memory. New York: 
Academic Press, 1972. 

Wandersman, L. P., & Wandersman, A. Strategy selection in concept identifi- 
cation under conditions varying cognitive load; A comparison of seventh 
grade and college students . Paper presented at the annual meeting of 
the Eastern Psychological Association, Washington, D. C. 1973. 

Wheaton, G- R., & Mirabella, A. Effects of task index variations on 

training effectiveness criteria . Technical Report No. NAVTRADEVCEN 
71-C-0059-1, 1972. Naval Training Device Center, Orlando, Florida. 

Wheaton, G. R., Mirabella, A., & Farina, A. J., Jr. Trainee and instructor 
task quantification: Development of quantitative indices and a pre- 
dictive methodology . Technical Report No. NAVTRADEVCEN 69-C-0278-1, 
January 1971. Naval Training Device Center, Orlando, Florida. 

Wheaton, G. R., Shaffer (Eisner), E., Mirabella, A., & Fleishman, E. A. 
Methods for predicting job-ability requirements; I. Ability 
requirements as a function of changes in the characteristics of an 
auditory signal identification task . Report No. AIR-31300-9/73-TR, 
September 1973, American Institutes for Research, Washington, fl. C. 

Zimmerman, W. S. The influence of item complexity upon the factor composi- 
tion of a spatial visualization test. Educational and Psychological 
Measurement , 1954, 14, 106-119. 



73 

83 



DISTRIBUTION LIST 



NAVY 



Dr. Marshall J. Farr, Director 
Personnel and Training Research 
Programs 

Office of Naval Research (Code 458) 
Arlington, VA 22217 

ONR Branch Office 
1030 East Green Street 
Pasadena, CA 91101 
ATTN; E. E. Gloye 

Director 

Naval Research Laboratory 
Code 2627 

Washington, DC 20390 

Special Assistant for Manpower 
OASN (M&RA) 
Pentagon, Room 4E794 
Washington, DC 20350 

Chief of Naval Reserve 
Code 3055 

New Orleans, I A 70146 

Dr. Lee Miller 

Naval Air Systems Command 

AIR-413E 

Washington, DC 20361 
Chief 

Bureau of Medicine & Surgery 
Research Division (Code 713) 
Washington, DC 20372 

Chief of Naval Education & Training 

Naval Air Station 

Pensacola, FL 32508 

ATTN: CAPT Bruce Stone, USN 

Mr. Arnold Rubinstein 
Naval Material Command (NAVMAT 03424) 
Room 820, Crystal Plaza #6 
Washington, DC 20360 



1 ONR Branch Office 
495 Summer Street 
Boston, MA 02210 
ATTN: Research Psychologist 

1 ONR Branch Office 

536 South Clark Street 
Chicago, IL 60605 

12 Defense Documentation Center 
Cameron Station, Building 5 
5010 Duke Street 
Alexandria, VA 22314 

1 LCDR Charles J. Theisen, Jr., 
MSC, USN 
4024 

Naval Air Development Center 
Warminster, PA 18974 

1 Navy Personnel Research 
and Development Center 
Code 9041 

San Diego, CA 92152 
ATTN: Dr. J. D. Fletcher 

1 Commanding Officer 

U.S. Naval Amphibious School 
Coronado, CA 92155 

1 Chairman 

Behavioral Science Department 
Naval Command & Management 
Division 
U.S. Naval Academy 
Luce Hall 

Annapolis, MD 21402 

1 Commanding Officer 

Naval Medical Neuropsychiatric 
Research Unit 
San Diego, CA 92152 



ERLC 



8'1 



Director, Navy Occupational Task 

Analysis Program (NOTAP) 
Navy Personnel Program Support 

Activity 

Building 1304, Boiling AFB 
Washington, DC 20336 

Dr. Robert Smith 

Chief of Naval Operations (0P-987E) 
Department of the Navy 
Washington, DC 20350 

Commander, Navy Recruiting Command 
4015 Wilson Boulevard 
Arlington, VA 22203 
ATTN: Code 015 

Chief of Naval Technical Training 
Naval Air Station Memphis (75) 
Millington, TN 38054 
ATTN: Dr. Norman J. Kerr 

Dr. Alfred F. Smode, Staff Consultant 
Training Analysis & Evaluation Group 
Naval Training Equipment Center 
Code N-OOT 
Orlando, PL 32813 

Dr. Martin Wiskoff 

Navy Personnel R&D Center 

San Diego, CA 92152 

Navy Personnel R&D Center ^ 
San Diego, CA 92152 
AHN: Code 10 



ARMY 

Headquarters 

U.S. Anny Administration Center 
Personnel Administration Combat 
Development Activity 
ATCP-HRO 

Ft. Benjamin Harrison, IN 46249 
Commandant 

United States Anny Infantry School 

ATTN: ATSH-DET 

Fort Benning, GA 31905 



1 Dr. Richard J. Niehaus 

Office of Civilian Manpower 
Management 
Code 06A 

Washington, DC 20390 



1 Superintendent 

Naval Postgraduate School 

Monterey, CA 93940 

ATTN: Library (Code 2124) 

1 Mr. George N. Graine 

Naval Ship Systems Command 
SHIPS 047C12 
Washington, DC 20362 

1 Dr. William L. Maloy 

Principal Civilian Advisor 

for Education & Training 
Naval Training Command, 

Code 01 A 

Pensacola, FL 32508 

1 Chief of Naval Training 
Support 
Code N-21 
Building 45 
Naval Air Station 
Pensacola, FL 32508 

1 D. M. Gragg, CAPT, MC, USN 
Head, Educational Programs 

Development Department 
Naval Health Sciences 

Education and Training Command 
Bethesda, MD 20014 



1 Armed Forces Staff College 
Norfolk, VA 23511 
AHN: Library 

1 Deputy Commander 

U.S. Army Institute of 
Administration 

Fort Benjamin Harrison, IN 46216 
ATTN: EA 



1 



1 



1 



1 



1 



1 



1 



1 



ERIC 



Dr. Stanley L. Cohen 
U.S. Army Research Institute 
1300 Wilson Boulevard 
Arlington, VA 22209 

Dr. J. E. Uhlaner, Technical Director 
U.S. Anny Research Institute 
1300 Wilson Boulevard 
Arlington, VA 22209 

HQ USAREUR & 7th Army 
ODCSOPS 

USAREUR Director of 6ED 
APO New York 09403 

AIR FORCE 

Research Branch 
AF/DPMYAR 

Randolph ARB, TX 78148 

AFHRL/DOJN 
Stop #63 

Lackland AFB, TX 78236 

Dr. Martin Rockway (AFHRL/TT) 
Lowry AFB 
Colorado 80230 

Commandant 

USAF School of Aerospace Medicine 
Aeromedical Library (SUL-4) 
Brooks AFB, TX 78235 

AFHRL/PE 
Stop 63 

Lackland AFB, TX 78236 

MARINE CORPS 

Mr. E. A. Dover 

Manpower Measurement Unit (Code MPI) 
Arlington Annex, Room 2413 
Arlington, VA 20380 

Director, Office of Manpower 
Utilization 

Headquarters, Marine Corps (Code MPU) 
MCB (Building 2009) 
Quantico, VA 22134 



1 Dr. Ralph Dusek 

U.S. Ariny Research Institute 
1300 Wilson Boulevard 
Arlington, VA 22209 

1 Dr. Milton S. Katz 

U.S. Artny Research Institute 
1300 Wilson Boulevard 
Arlington, VA 22209 



1 Dr. 6. A. Eckstrand (AFHRL/AS) 
Wright-Patterson AFB 
Ohio 45433 

1 Dr. Robert A. Bottenberg 
(AFHRL/SM) 
Stop #63 

Lackland AFB, TX 78236 

1 AFOSR/NL 

1400 Wilson Boulevard 
Arlington, VA 22209 

1 CAPT Jack Thorpe, USAF 

Flying Training Division (HRL) 
Williams AFB, AZ 85224 



1 Commandant of the Marine Corps 
Headquarters, U.S. Marine Corps 
Code MPI-20 
Washington, DC 20380 

1 Dr. A. L. Slafkosky 

Scientific Advisor (Code RD-1) 
Headquarters, U.S. Marine Corps 
Washington, DC 20380 



8S 



COAST GUARD 



Mr. Joseph J. Cowan, Chief 
Psychological Research Branch (G-P-.1/62) 
U.S. Coast Guard Headquarters 
Washington, D. C. 20590 

OTHER DOD 

Lt. Col. Henry L. Taylor, USAF 
Military Assistant for Human 
Resources 
OAD (E&LS) ODDR^E 
Pentagon, Room 3D129 
Washington, DC 20301 

Dr. Harold O'Neil, Jr. 
Advanced Research Projects Agency 
Human Resources Research Office 
1400 Wilson Boulevard, Room 625 
Arlington, VA 22209 



OTHER GOVERNMENT 

Dr. Lorraine D. Eyde 
Personnel Research and Development 
Center 

U.S. Civil Service Commission 
1900 £ Street, N.W. 
Washington, DC 20415 

Dr. Vern Urry 

Personnel Research and Development 
Center 

U.S. Civil Service Commission 
1900 E Street, N.W. 
Washington, DC 20415 

U.S. Civil Service Commission 
Federal Office Bldg. 
Chicago Regional Staff Div. 
ATTN: C.S. Winiewicz 
Regional Psychologist - 
230 So. Dearborn St. 
Chicago, IL 60604 

MISCELLANEOUS 

Dr. Scarvia B. Anderson 
Educational Testing Service 
17 Executive Park Drive, N.E. 
Atlanta, GA 30329 



Mr. William J, Stormer 
DOD Computer Institute 
Washington Navy Yard, 
Building 175 
Washington, DC 20374 



Helga L. Yeich 
Advanced Research Projects 
Agency 

Manpower Management Office 
1400 Wilson Boulevard 
Arlington, VA 22209 



Dr. William Gorham, Director 
Personnel Research and 
Development Center 
U.S. Civil Service Commission 
1900 E Street, N.W. 
Washington, DC 20415 

Dr. Andrew R. Molnar 
Technological Innovations 

in Education 
National Science Foundation 
Washington, DC 20550 



Dr. Richard Snow 
Stanford University 
School of Education 
Stanford, CA 94305 



87 



1 



1 



1 



1 



1 



1 



1 



1 



1 



ERIC 



Dr- John Annett 
The Open University 
Milton Keynes 
Buckinghamshire 
ENGLAND 

Dr* Bernard Bass 
University of Rochester 
Management Research Center 
Rochester, NY 14627 

Mr* Kenneth M* Bromberg 
Manager - Washington Operations 
Information Concepts, Inc* 
1701 North Fort Myer Drive 
Arlington, VA 22209 

Dr* Ronald P. Carver 

School of Education 

University of Missouri - Kansas City 

Kansas City, Missouri 64110 

Dr. Kenneth E. Clark 
University of Rochester 
College of Arts & Sciences 
River Campus Station 
Rochester, NY 14627 

Dr* Norman R* Dixon 
Room 170 

190 Lothrop Street 
Pittsburgh, PA 15260 

Dr. Marvin D* Dunnette 
University of Minnesota 
Department of Psychology 
Minneapolis, MN 55455 

Dr. Victor Fields 
Montgomery College 
Department of Psychology 
Rockville, MD 20850 

Dr. Robert Glaser, Director 
University of Pittsburgh 
Learning Research & Development Center 
Pittsburgh, PA 15213 

LCol CRJ Lafleur, Director 
Personnel Applied Research 
National Defence HQ 
Ottawa, Canada KIA OK 2 



1 Dr. Gerald V. Barrett 
University of Akron 
Department of Psychology 
Akron, OH 44325 



1 Dr. David G. Bowers 
University of Michigan 
Institute for Social Research 
Ann Arbor, MI 48106 

1 Dr. Norman Cliff 

University of Southern 
California 

Departp:ent of Psychology 

University Park 

Los Angelas, CA 90007 

1 Century Research Corporation 
4113 Lee Highway 
Arlington, VA 22207 

1 Dr. Rene^ V. Dawis 

University of Minnesota 
Department of Psychology 
Minneapolis, MN 55455 



1 Dr. Robert Dubin 

University of California 
Graduate School of Administration 
Irvine, CA 92664 

1 ERic 

Processing and Reference Facility 
4833 Rugby Avenue 
Bethesda, MD 20014 

1 Dr. Ruth Day 
Yale University 
Department of Psychology 
New Haven, CT 06520 

1 Dr. Robert Vineberg 
HumRRO Western Division 
27857 Berwick Drive 
Carmel, CA 93921 

1 Mr. Harry H. Harman 

Educational Testing Service 
Princeton, NJ 08540 



8S 



' Dr. Richard S. Hatch 

Decision Systems Associates., Inc. 
11428 Rockville Pike 
Rockville, MD 20852 

HumRRO 

Division No. 3 
P. 0. Box 5787 

Presidio of Monterey, CA 93940 
HumRRO 

Division No. 6, Library 

P. 0. Box 428 

Fort Rucker, AL 36360 

Dr. Steven W. Keele 
University of Oregon 
Department of Psychology 
Eugene, OR 97403 

Dr. Frederick M. Lord 
Educational Testing Service 
Princeton, NJ 08540 

Dr. Robert R. Mackie 

Human Factors Research, Inc. 

6780 Cortona Drive 

Santa Barbara Research Park 

Go! eta, CA 93017 

Dr. Donald A. Norman 
University of California, San Diego 
Center for Human Information Processing 
LaJolla, CA 92037 

Dr. Joseph W. Rigney 

University of Southern California 

Behavioral Technology Laboratories 

3717 South Grand 

Los Angeles, CA 90007 

Dr. George E. Rowland 
Rowland and Company, Inc. 
P. 0. Box 61 
Haddonfield, NJ 08033 

Dr. C. Harold Stone 
1428 Virginia Avenue 
Glendale, CA 91202 

Dr. Benton J. Underwood 
Northwestern University 
^ Department of Psychology 
Fvanston, IL 60201 



Dr. M. D. Havron 
Human Sciences Research, Inc. 
7710 Old Spring House Road 
West Gate Industrial Park 
McLean, VA 22101 

HumRRO 

Division No. 4, Infantry 

P. 0. Box 2086 

Fort Benning, GA 31905 

Dr. Lawrence B. Johnson 
Lawrence Johnson & Associates, 
Inc. 

200 S Street, N.W., Suite 502 
Washington, DC 20009 

Dr. David Klahr 
Carnegie-Mellon University 
Department of Psychology 
Pittsburgh, PA 15213 

Dr. Ernest J. McCormick 
Purdue University 
Department of Psychological 
Sciences 

Lafayette, IN 47907 

Dr. Leo Munday, Vice-President 
American College Testing Program 
P. 0. Box 168 
Iowa City, lA 52240 

Dr. Diane M. Ramsey-Klee 
R-K Research & System Design 
3947 Ridgemont Drive 
Malibu, CA 90265 

Dr. Leonard L. Rosenbaum 
Chairman 

Montgomery College 
Department of Psychology 
Rockville, MD 20850 

Dr. Arthur I. Siegel" 
Applied Psychological Services^ 
404 East Lancaster Avenue 
Wayne, PA 19087 

Dr. David J. Weiss 
University of Minnesota 
Department of Psychology 
Minneapolis, MN 55455 



ERIC 



89 



